2024-09-22T03:00:00.671 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-22T03:00:00.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T03:00:00.774 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732 branch: squid description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914732' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore rocksdb cf: false crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 err to stderr: true log flush on exit: true log to file: false log to stderr: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: bluestore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: min_in: 4 workunit: branch: squid env: CRIMSON_COMPAT: '1' sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 1158 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 0/1 suite: crimson-rados suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJQlVPTXtg49Z8Acf51W4FW0JfXcXg7psEnOFhZars3BzTz/AMDpjoCVvpyMe3/DKrZLBFzEk7f/bjjJRiTdZqk= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCiwpGJfMU0EBteNF0PggHbh+k1vMXFqSA+IdXAHwMhJLg9EYKEozrW3AP2xq0JknUfFzJSCRj/9FNbc2EofzXg= tasks: - install: null - ceph: conf: osd: debug monc: 20 flavor: crimson - thrashosds: ceph_objectstore_tool: false chance_inject_pause_short: 0 chance_reset_purged_snaps_last: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:56:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-22T03:00:00.774 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-22T03:00:00.775 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-22T03:00:00.775 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-22T03:00:00.777 INFO:teuthology.task.internal:Checking packages... 2024-09-22T03:00:00.798 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-22T03:00:00.799 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-22T03:00:00.799 INFO:teuthology.packaging:ref: None 2024-09-22T03:00:00.799 INFO:teuthology.packaging:tag: None 2024-09-22T03:00:00.799 INFO:teuthology.packaging:branch: squid 2024-09-22T03:00:00.799 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:00:00.799 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-22T03:00:00.997 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373.g4e45e6b4 2024-09-22T03:00:00.999 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-22T03:00:01.005 INFO:teuthology.task.internal:no buildpackages task found 2024-09-22T03:00:01.005 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-22T03:00:01.049 INFO:teuthology.task.internal:Saving configuration 2024-09-22T03:00:01.082 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-22T03:00:01.107 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-22T03:00:01.133 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 02:54:51.207047', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJQlVPTXtg49Z8Acf51W4FW0JfXcXg7psEnOFhZars3BzTz/AMDpjoCVvpyMe3/DKrZLBFzEk7f/bjjJRiTdZqk='} 2024-09-22T03:00:01.158 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 02:54:51.208588', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCiwpGJfMU0EBteNF0PggHbh+k1vMXFqSA+IdXAHwMhJLg9EYKEozrW3AP2xq0JknUfFzJSCRj/9FNbc2EofzXg='} 2024-09-22T03:00:01.158 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-22T03:00:01.212 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-09-22T03:00:01.212 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-22T03:00:01.212 INFO:teuthology.run_tasks:Running task console_log... 2024-09-22T03:00:01.341 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3a56810160>, signals=[15]) 2024-09-22T03:00:01.341 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-22T03:00:01.346 INFO:teuthology.task.internal:Opening connections... 2024-09-22T03:00:01.346 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-09-22T03:00:01.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:00:01.422 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-09-22T03:00:01.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:00:01.496 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-22T03:00:01.501 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-09-22T03:00:01.520 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-09-22T03:00:01.520 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-09-22T03:00:01.578 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-09-22T03:00:01.578 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-09-22T03:00:01.578 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-09-22T03:00:01.578 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-09-22T03:00:01.578 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-09-22T03:00:01.578 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-22T03:00:01.579 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-22T03:00:01.580 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-09-22T03:00:01.604 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-09-22T03:00:01.623 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-09-22T03:00:01.623 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-09-22T03:00:01.681 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-22T03:00:01.682 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-22T03:00:01.683 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-22T03:00:01.683 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-09-22T03:00:01.704 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-22T03:00:01.712 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-22T03:00:01.735 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-22T03:00:01.735 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-09-22T03:00:01.738 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-09-22T03:00:01.755 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-22T03:00:01.822 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-22T03:00:01.822 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-09-22T03:00:01.825 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-09-22T03:00:01.943 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-22T03:00:02.003 INFO:teuthology.run_tasks:Running task kernel... 2024-09-22T03:00:02.025 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-22T03:00:02.025 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-22T03:00:02.026 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T03:00:02.026 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T03:00:02.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:00:02.043 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-22T03:00:02.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:00:02.044 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-22T03:00:02.102 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-503.el9.x86_64 2024-09-22T03:00:02.102 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-503.el9.x86_64 2024-09-22T03:00:02.102 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-09-22T03:00:02.105 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-503.el9.x86_64 2024-09-22T03:00:02.106 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.14.0-503.el9.x86_64 2024-09-22T03:00:02.106 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum install -y kernel 2024-09-22T03:00:04.419 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-22T03:00:06.120 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-22T03:00:06.538 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.3 MB 00:02 2024-09-22T03:00:09.749 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 20 MB 00:02 2024-09-22T03:00:11.879 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-22T03:00:13.355 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-22T03:00:13.931 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-22T03:00:14.517 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-22T03:00:15.440 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-22T03:00:16.586 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-22T03:00:19.923 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-22T03:00:21.805 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-22T03:00:21.805 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-22T03:00:21.910 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:00:21.919 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:00:21.919 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-22T03:00:21.919 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:00:21.919 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:00:21.919 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout:Install 4 Packages 2024-09-22T03:00:21.920 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:00:21.921 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 79 M 2024-09-22T03:00:21.921 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 126 M 2024-09-22T03:00:21.921 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:00:22.066 INFO:teuthology.orchestra.run.smithi146.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 291 kB/s | 36 kB 00:00 2024-09-22T03:00:22.241 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-22T03:00:24.148 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-22T03:00:24.148 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-22T03:00:24.244 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-22T03:00:24.254 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:00:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:00:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:00:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-09-22T03:00:24.255 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:00:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 79 M 2024-09-22T03:00:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 126 M 2024-09-22T03:00:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:00:24.299 INFO:teuthology.orchestra.run.smithi146.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-22T03:00:24.626 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 381 kB/s | 36 kB 00:00 2024-09-22T03:00:25.010 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-22T03:00:25.377 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-22T03:00:25.687 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-09-22T03:00:25.688 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:00:25.688 INFO:teuthology.orchestra.run.smithi148.stdout:Total 55 MB/s | 79 MB 00:01 2024-09-22T03:00:26.366 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:00:26.702 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:00:26.702 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:00:27.654 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:00:27.654 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:00:29.082 INFO:teuthology.orchestra.run.smithi146.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-09-22T03:00:30.093 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:00:30.734 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:00:30.827 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:00:33.584 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:00:33.720 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:00:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 2.1 MB/s | 35 MB 00:16 2024-09-22T03:00:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:00:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Total 4.8 MB/s | 79 MB 00:16 2024-09-22T03:00:38.705 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:00:38.737 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:00:38.904 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:00:39.284 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:00:39.284 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:00:40.272 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:00:40.272 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:00:42.709 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:00:43.349 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:00:43.440 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:00:43.562 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:00:46.207 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:00:46.344 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:00:51.149 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:00:51.185 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:00:56.001 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:22.712 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:22.712 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:01:22.712 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:01:22.712 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:01:23.575 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:01:23.910 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:01:23.991 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:01:23.991 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:01:23.991 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-22T03:01:23.991 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T03:01:23.991 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T03:01:23.991 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-22T03:01:23.991 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T03:01:23.992 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:01:24.050 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:01:24.051 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:01:24.051 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-09-22T03:01:24.086 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-09-22T03:01:24.086 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-09-22T03:01:24.086 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-09-22T03:01:24.086 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 82.el9 2024-09-22T03:01:24.086 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8274670 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:01:24.087 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-22T03:01:24.088 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-22T03:01:24.088 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-22T03:01:24.088 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-09-22T03:01:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:01:24.088 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-09-22T03:01:24.089 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-22T03:01:24.089 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-508.el9.x86_64 2024-09-22T03:01:24.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-22T03:01:24.735 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-09-22T03:01:25.983 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-22T03:01:26.019 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-09-22T03:01:26.022 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-22T03:01:26.038 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.UPW4dC0olh 2024-09-22T03:01:26.038 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UPW4dC0olh 2024-09-22T03:01:26.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.UPW4dC0olh 2024-09-22T03:01:28.590 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.UPW4dC0olh is 6KB 2024-09-22T03:01:28.614 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.UPW4dC0olh 2024-09-22T03:01:28.631 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-09-22T03:01:28.696 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-22T03:01:28.697 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-22T03:01:28.697 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-22T03:01:28.697 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-22T03:01:28.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-22T03:01:28.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-09-22T03:01:34.283 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:34.323 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:34.664 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:34.664 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:01:34.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:01:34.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:01:35.530 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:01:35.912 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:01:35.993 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:01:35.994 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:01:35.994 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-22T03:01:35.994 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T03:01:35.994 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T03:01:35.995 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-22T03:01:35.995 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T03:01:35.995 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:01:36.054 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:01:36.055 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:01:36.055 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -qi grub2-tools 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Name : grub2-tools 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Epoch : 1 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Version : 2.06 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Release : 82.el9 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: x86_64 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Group : Unspecified 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Size : 8274670 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:License : GPLv3+ 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-22T03:01:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:Packager : builder@centos.org 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:Vendor : CentOS 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:Summary : Support tools for GRUB. 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:Description : 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-22T03:01:36.088 INFO:teuthology.orchestra.run.smithi146.stdout:hardware devices. 2024-09-22T03:01:36.089 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:01:36.089 INFO:teuthology.orchestra.run.smithi146.stdout:This subpackage provides tools for support of all platforms. 2024-09-22T03:01:36.089 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-22T03:01:36.090 INFO:teuthology.task.kernel:Updating grub on smithi146 to boot 5.14.0-508.el9.x86_64 2024-09-22T03:01:36.090 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-22T03:01:36.726 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2024-09-22T03:01:38.077 INFO:teuthology.orchestra.run.smithi146.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-22T03:01:38.113 INFO:teuthology.orchestra.run.smithi146.stderr:done 2024-09-22T03:01:38.115 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-22T03:01:38.131 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.EaWbIHTRLR 2024-09-22T03:01:38.131 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EaWbIHTRLR 2024-09-22T03:01:38.197 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.EaWbIHTRLR 2024-09-22T03:01:38.349 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.EaWbIHTRLR is 6KB 2024-09-22T03:01:38.371 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.EaWbIHTRLR 2024-09-22T03:01:38.387 DEBUG:teuthology.orchestra.run.smithi146:> sudo /bin/ls /boot/loader/entries || true 2024-09-22T03:01:38.450 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-22T03:01:38.450 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-22T03:01:38.450 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-22T03:01:38.450 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-22T03:01:38.452 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-22T03:01:38.600 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2024-09-22T03:01:58.878 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-22T03:01:58.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-22T03:01:58.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:02:08.610 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-22T03:02:08.611 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-22T03:02:08.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:02:17.327 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-22T03:02:26.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-22T03:02:26.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:02:27.055 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-22T03:02:29.395 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-22T03:02:36.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-22T03:02:36.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:02:39.121 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-22T03:02:41.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-22T03:02:41.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:02:51.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-22T03:02:51.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:02:59.823 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-22T03:03:09.551 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-22T03:03:14.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-22T03:03:14.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:03:15.194 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-22T03:03:15.575 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-22T03:03:15.575 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-508.el9.x86_64"... 2024-09-22T03:03:15.575 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-22T03:03:15.589 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-508.el9.x86_64 2024-09-22T03:03:15.590 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-22T03:03:15.590 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T03:03:15.590 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-22T03:03:16.591 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-22T03:03:16.591 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T03:03:16.686 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-09-22T03:03:16.703 DEBUG:teuthology.parallel:result is None 2024-09-22T03:03:24.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-22T03:03:24.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:03:24.960 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-22T03:03:25.361 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-22T03:03:25.361 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-22T03:03:25.361 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-22T03:03:25.376 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-508.el9.x86_64 2024-09-22T03:03:25.376 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-22T03:03:25.376 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T03:03:25.376 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-22T03:03:26.377 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-22T03:03:26.377 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T03:03:26.454 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2024-09-22T03:03:26.486 DEBUG:teuthology.parallel:result is None 2024-09-22T03:03:26.486 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-22T03:03:26.492 INFO:teuthology.task.internal:Creating test directory... 2024-09-22T03:03:26.493 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T03:03:26.495 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T03:03:26.515 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-22T03:03:26.551 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-22T03:03:26.583 INFO:teuthology.task.internal:Creating archive directory... 2024-09-22T03:03:26.584 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T03:03:26.586 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T03:03:26.639 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-22T03:03:26.645 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-22T03:03:26.645 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T03:03:26.647 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T03:03:26.691 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:03:26.702 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:03:26.708 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:03:26.719 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:03:26.720 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-22T03:03:26.727 INFO:teuthology.task.internal:Configuring sudo... 2024-09-22T03:03:26.727 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T03:03:26.729 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T03:03:26.771 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-22T03:03:26.779 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-22T03:03:26.779 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T03:03:26.798 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T03:03:26.828 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:03:26.896 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:03:26.943 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:03:27.027 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:03:27.073 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:03:27.073 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T03:03:27.142 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:03:27.188 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:03:27.244 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:03:27.326 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:03:27.372 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:03:27.373 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T03:03:27.440 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-09-22T03:03:27.442 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-09-22T03:03:27.478 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:03:27.527 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:03:27.973 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-22T03:03:27.981 INFO:teuthology.task.internal:Starting timer... 2024-09-22T03:03:27.981 INFO:teuthology.run_tasks:Running task pcp... 2024-09-22T03:03:28.011 INFO:teuthology.run_tasks:Running task selinux... 2024-09-22T03:03:28.044 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-22T03:03:28.044 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-09-22T03:03:28.126 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-09-22T03:03:28.128 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-09-22T03:03:28.176 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-09-22T03:03:28.178 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-22T03:03:28.178 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-09-22T03:03:28.196 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-09-22T03:03:28.196 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-09-22T03:03:28.251 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-09-22T03:03:28.251 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi146.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-09-22T03:03:28.252 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:03:28.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:03:28.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:03:28.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:03:28.306 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-22T03:03:28.306 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-09-22T03:03:28.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-09-22T03:03:28.398 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-22T03:03:28.407 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-22T03:03:28.416 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-22T03:03:28.417 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi146.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-22T03:07:11.951 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi146.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-09-22T03:07:11.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-22T03:07:11.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:07:12.037 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-22T03:07:12.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-22T03:07:12.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-22T03:07:12.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:07:12.200 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-22T03:07:12.278 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-22T03:07:12.278 INFO:teuthology.run_tasks:Running task clock... 2024-09-22T03:07:12.295 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-22T03:07:12.295 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T03:07:12.296 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:07:12.298 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T03:07:12.298 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:07:12.325 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-22T03:07:12.339 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-22T03:07:12.348 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-22T03:07:12.362 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-22T03:07:12.364 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-09-22T03:07:12.375 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-09-22T03:07:12.387 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-09-22T03:07:12.389 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-22T03:07:12.398 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-09-22T03:07:12.403 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-22T03:07:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-22T03:07:12.427 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-22T03:07:12.455 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:07:12.459 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:07:12.459 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-22T03:07:12.459 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.459 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.459 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.459 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.482 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:07:12.486 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:07:12.486 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-22T03:07:12.486 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.486 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.486 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.486 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:07:12.487 INFO:teuthology.run_tasks:Running task install... 2024-09-22T03:07:12.495 DEBUG:teuthology.task.install:project ceph 2024-09-22T03:07:12.495 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T03:07:12.495 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-22T03:07:12.495 INFO:teuthology.task.install:Using flavor: crimson 2024-09-22T03:07:12.501 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-22T03:07:12.501 INFO:teuthology.task.install:extra packages: [] 2024-09-22T03:07:12.501 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'tag': None, 'wait_for_package': False} 2024-09-22T03:07:12.501 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:07:12.503 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'tag': None, 'wait_for_package': False} 2024-09-22T03:07:12.503 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:07:12.644 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/ 2024-09-22T03:07:12.645 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-22T03:07:12.649 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/ 2024-09-22T03:07:12.649 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-22T03:07:12.755 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-22T03:07:12.756 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:07:12.756 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-22T03:07:12.761 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-22T03:07:12.761 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:07:12.761 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-22T03:07:12.782 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-22T03:07:12.783 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e45e6b490e6cbe082cdf155fea780438ec53fc7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-22T03:07:12.787 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-22T03:07:12.787 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e45e6b490e6cbe082cdf155fea780438ec53fc7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-22T03:07:12.849 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-22T03:07:12.853 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-22T03:07:12.923 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-22T03:07:12.928 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-22T03:07:12.987 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-09-22T03:07:12.989 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-09-22T03:07:12.992 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-09-22T03:07:12.994 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-09-22T03:07:13.214 INFO:teuthology.orchestra.run.smithi146.stdout:45 files removed 2024-09-22T03:07:13.216 INFO:teuthology.orchestra.run.smithi148.stdout:45 files removed 2024-09-22T03:07:13.241 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw 2024-09-22T03:07:13.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw 2024-09-22T03:07:13.803 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 316 kB/s | 87 kB 00:00 2024-09-22T03:07:13.808 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 315 kB/s | 87 kB 00:00 2024-09-22T03:07:14.079 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-22T03:07:14.093 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-22T03:07:14.338 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 8.9 kB/s | 2.0 kB 00:00 2024-09-22T03:07:14.368 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 8.3 kB/s | 2.0 kB 00:00 2024-09-22T03:07:14.879 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-22T03:07:17.751 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.3 MB 00:03 2024-09-22T03:07:19.597 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 20 MB 00:03 2024-09-22T03:07:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-22T03:07:23.831 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-22T03:07:24.245 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-22T03:07:25.298 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 19 kB 00:00 2024-09-22T03:07:25.779 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 95 kB/s | 19 kB 00:00 2024-09-22T03:07:26.811 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-22T03:07:27.042 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-22T03:07:32.200 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-22T03:07:32.689 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-22T03:07:34.452 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:07:34.459 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:07:34.459 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:07:34.459 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:07:34.459 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:07:34.459 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 9.9 M 2024-09-22T03:07:34.459 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-09-22T03:07:34.459 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 19 M 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 760 k 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-22T03:07:34.460 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 501 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.1 M 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 161 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 301 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-22T03:07:34.461 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout:Install 30 Packages 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 2 Packages 2024-09-22T03:07:34.462 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:34.463 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 58 M 2024-09-22T03:07:34.463 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:07:34.949 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:07:34.962 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 9.9 M 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 19 M 2024-09-22T03:07:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 760 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 501 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.1 M 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-22T03:07:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 161 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 301 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:07:34.965 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:07:34.966 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:07:34.966 INFO:teuthology.orchestra.run.smithi148.stdout:Install 30 Packages 2024-09-22T03:07:34.966 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2024-09-22T03:07:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:07:34.966 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 58 M 2024-09-22T03:07:34.966 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:07:35.160 INFO:teuthology.orchestra.run.smithi146.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 13 MB/s | 5.4 MB 00:00 2024-09-22T03:07:35.210 INFO:teuthology.orchestra.run.smithi146.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 504 kB/s | 25 kB 00:00 2024-09-22T03:07:35.293 INFO:teuthology.orchestra.run.smithi146.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 8.9 MB/s | 760 kB 00:00 2024-09-22T03:07:35.368 INFO:teuthology.orchestra.run.smithi146.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 6.5 MB/s | 501 kB 00:00 2024-09-22T03:07:35.611 INFO:teuthology.orchestra.run.smithi146.stdout:(5/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 12 MB/s | 9.9 MB 00:00 2024-09-22T03:07:35.703 INFO:teuthology.orchestra.run.smithi146.stdout:(6/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_64 15 MB/s | 5.1 MB 00:00 2024-09-22T03:07:35.728 INFO:teuthology.orchestra.run.smithi146.stdout:(7/32): python3-ceph-argparse-19.1.1-373.g4e45e 387 kB/s | 45 kB 00:00 2024-09-22T03:07:35.761 INFO:teuthology.orchestra.run.smithi146.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 2.3 MB/s | 136 kB 00:00 2024-09-22T03:07:35.786 INFO:teuthology.orchestra.run.smithi146.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 2.7 MB/s | 161 kB 00:00 2024-09-22T03:07:35.820 INFO:teuthology.orchestra.run.smithi146.stdout:(10/32): python3-rados-19.1.1-373.g4e45e6b4.el9 5.3 MB/s | 321 kB 00:00 2024-09-22T03:07:35.845 INFO:teuthology.orchestra.run.smithi146.stdout:(11/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-22T03:07:35.871 INFO:teuthology.orchestra.run.smithi146.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-22T03:07:36.096 INFO:teuthology.orchestra.run.smithi146.stdout:(13/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x 14 MB/s | 19 MB 00:01 2024-09-22T03:07:36.146 INFO:teuthology.orchestra.run.smithi146.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 356 kB/s | 107 kB 00:00 2024-09-22T03:07:36.172 INFO:teuthology.orchestra.run.smithi146.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 533 kB/s | 160 kB 00:00 2024-09-22T03:07:36.197 INFO:teuthology.orchestra.run.smithi146.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-22T03:07:36.264 INFO:teuthology.orchestra.run.smithi146.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-22T03:07:36.291 INFO:teuthology.orchestra.run.smithi146.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 449 kB/s | 42 kB 00:00 2024-09-22T03:07:36.325 INFO:teuthology.orchestra.run.smithi146.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.6 MB/s | 662 kB 00:00 2024-09-22T03:07:36.350 INFO:teuthology.orchestra.run.smithi146.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.5 MB/s | 221 kB 00:00 2024-09-22T03:07:36.375 INFO:teuthology.orchestra.run.smithi146.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 428 kB/s | 36 kB 00:00 2024-09-22T03:07:36.401 INFO:teuthology.orchestra.run.smithi146.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 625 kB/s | 47 kB 00:00 2024-09-22T03:07:36.459 INFO:teuthology.orchestra.run.smithi146.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-22T03:07:36.485 INFO:teuthology.orchestra.run.smithi146.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 373 kB/s | 26 kB 00:00 2024-09-22T03:07:36.510 INFO:teuthology.orchestra.run.smithi146.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-22T03:07:36.535 INFO:teuthology.orchestra.run.smithi146.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-22T03:07:36.569 INFO:teuthology.orchestra.run.smithi146.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-22T03:07:36.602 INFO:teuthology.orchestra.run.smithi146.stdout:(28/32): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-22T03:07:36.661 INFO:teuthology.orchestra.run.smithi146.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-22T03:07:36.665 INFO:teuthology.orchestra.run.smithi148.stdout:(1/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 6.7 MB/s | 9.9 MB 00:01 2024-09-22T03:07:36.753 INFO:teuthology.orchestra.run.smithi146.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-22T03:07:36.765 INFO:teuthology.orchestra.run.smithi148.stdout:(2/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 3.4 MB/s | 5.4 MB 00:01 2024-09-22T03:07:36.790 INFO:teuthology.orchestra.run.smithi148.stdout:(3/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 201 kB/s | 25 kB 00:00 2024-09-22T03:07:36.853 INFO:teuthology.orchestra.run.smithi146.stdout:(31/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-22T03:07:36.882 INFO:teuthology.orchestra.run.smithi148.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 5.3 MB/s | 501 kB 00:00 2024-09-22T03:07:36.945 INFO:teuthology.orchestra.run.smithi146.stdout:(32/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 11 MB/s | 3.1 MB 00:00 2024-09-22T03:07:36.949 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:07:36.950 INFO:teuthology.orchestra.run.smithi146.stdout:Total 23 MB/s | 58 MB 00:02 2024-09-22T03:07:36.982 INFO:teuthology.orchestra.run.smithi148.stdout:(5/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 3.4 MB/s | 760 kB 00:00 2024-09-22T03:07:37.042 INFO:teuthology.orchestra.run.smithi148.stdout:(6/32): python3-ceph-argparse-19.1.1-373.g4e45e 776 kB/s | 45 kB 00:00 2024-09-22T03:07:37.047 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:07:37.084 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:07:37.084 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:07:37.225 INFO:teuthology.orchestra.run.smithi148.stdout:(7/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x8 9.6 MB/s | 19 MB 00:02 2024-09-22T03:07:37.250 INFO:teuthology.orchestra.run.smithi148.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 652 kB/s | 136 kB 00:00 2024-09-22T03:07:37.275 INFO:teuthology.orchestra.run.smithi148.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 3.2 MB/s | 161 kB 00:00 2024-09-22T03:07:37.326 INFO:teuthology.orchestra.run.smithi148.stdout:(10/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 5.9 MB/s | 301 kB 00:00 2024-09-22T03:07:37.351 INFO:teuthology.orchestra.run.smithi148.stdout:(11/32): python3-rados-19.1.1-373.g4e45e6b4.el9 3.1 MB/s | 321 kB 00:00 2024-09-22T03:07:37.376 INFO:teuthology.orchestra.run.smithi148.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-22T03:07:37.468 INFO:teuthology.orchestra.run.smithi148.stdout:(13/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_6 8.7 MB/s | 5.1 MB 00:00 2024-09-22T03:07:37.543 INFO:teuthology.orchestra.run.smithi148.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 557 kB/s | 107 kB 00:00 2024-09-22T03:07:37.602 INFO:teuthology.orchestra.run.smithi148.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-22T03:07:37.627 INFO:teuthology.orchestra.run.smithi148.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 639 kB/s | 160 kB 00:00 2024-09-22T03:07:37.686 INFO:teuthology.orchestra.run.smithi148.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-22T03:07:37.713 INFO:teuthology.orchestra.run.smithi148.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 492 kB/s | 42 kB 00:00 2024-09-22T03:07:37.738 INFO:teuthology.orchestra.run.smithi148.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-22T03:07:37.739 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:07:37.739 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:07:37.764 INFO:teuthology.orchestra.run.smithi148.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.8 MB/s | 221 kB 00:00 2024-09-22T03:07:37.789 INFO:teuthology.orchestra.run.smithi148.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 476 kB/s | 36 kB 00:00 2024-09-22T03:07:37.814 INFO:teuthology.orchestra.run.smithi148.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-22T03:07:37.873 INFO:teuthology.orchestra.run.smithi148.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-22T03:07:37.890 INFO:teuthology.orchestra.run.smithi148.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 432 kB/s | 26 kB 00:00 2024-09-22T03:07:37.915 INFO:teuthology.orchestra.run.smithi148.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-22T03:07:37.932 INFO:teuthology.orchestra.run.smithi148.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-22T03:07:37.974 INFO:teuthology.orchestra.run.smithi148.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-22T03:07:37.999 INFO:teuthology.orchestra.run.smithi148.stdout:(28/32): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-22T03:07:38.058 INFO:teuthology.orchestra.run.smithi148.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-22T03:07:38.166 INFO:teuthology.orchestra.run.smithi148.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-22T03:07:38.283 INFO:teuthology.orchestra.run.smithi148.stdout:(31/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 14 MB/s | 3.1 MB 00:00 2024-09-22T03:07:38.667 INFO:teuthology.orchestra.run.smithi148.stdout:(32/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 4.8 MB/s | 3.2 MB 00:00 2024-09-22T03:07:38.672 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:07:38.672 INFO:teuthology.orchestra.run.smithi148.stdout:Total 16 MB/s | 58 MB 00:03 2024-09-22T03:07:38.782 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:07:38.818 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:07:38.818 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:07:39.228 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:07:39.474 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:07:39.474 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:07:39.600 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-22T03:07:39.705 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:07:39.797 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:07:39.896 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:07:39.923 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:07:39.983 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:07:40.011 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-09-22T03:07:40.044 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-09-22T03:07:40.067 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:07:40.089 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:07:40.277 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:07:40.305 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:07:40.335 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-09-22T03:07:40.378 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 10/34 2024-09-22T03:07:40.407 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:07:40.483 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 12/34 2024-09-22T03:07:40.512 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:07:40.601 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:07:40.662 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-09-22T03:07:40.774 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-22T03:07:40.821 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-22T03:07:40.871 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-22T03:07:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-22T03:07:40.929 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-22T03:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-22T03:07:41.302 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-22T03:07:41.344 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:07:41.355 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-22T03:07:41.419 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:07:41.435 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:07:41.466 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-22T03:07:41.510 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-22T03:07:41.515 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:07:41.602 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:07:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:07:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:07:41.741 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-09-22T03:07:41.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-09-22T03:07:41.789 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:07:41.812 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:07:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:07:42.007 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-22T03:07:42.011 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:07:42.030 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:07:42.040 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-09-22T03:07:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 10/34 2024-09-22T03:07:42.093 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:07:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:07:42.171 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-22T03:07:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 12/34 2024-09-22T03:07:42.207 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-22T03:07:42.218 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:07:42.307 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:07:42.342 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-09-22T03:07:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-22T03:07:42.485 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-22T03:07:42.526 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-22T03:07:42.558 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-22T03:07:42.588 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-22T03:07:42.620 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-22T03:07:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-22T03:07:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:07:43.099 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:07:43.129 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-22T03:07:43.156 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-22T03:07:43.653 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-22T03:07:43.693 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:07:43.765 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:07:43.841 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-22T03:07:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-22T03:07:44.454 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:07:44.520 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:07:45.114 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:07:45.157 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:07:45.232 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:07:45.232 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-22T03:07:45.232 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-22T03:07:45.233 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:45.266 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:07:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:07:46.217 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:07:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:07:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:07:46.954 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:07:46.954 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-22T03:07:46.955 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-22T03:07:46.955 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:07:46.984 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:07:55.320 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:07:55.320 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-09-22T03:07:55.321 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-09-22T03:07:55.321 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-09-22T03:07:55.321 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-09-22T03:07:55.321 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-09-22T03:07:55.321 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-09-22T03:07:55.321 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-09-22T03:07:55.321 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:56.315 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:07:56.341 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:07:56.341 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:07:56.341 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-22T03:07:56.341 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:07:56.342 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:07:56.342 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:56.343 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:07:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:07:56.442 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-09-22T03:07:56.772 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:07:57.634 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:07:57.634 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-22T03:07:57.634 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:07:57.634 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:07:57.634 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:07:57.634 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-22T03:07:57.635 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-22T03:07:57.636 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:07:57.637 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-22T03:07:57.637 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-22T03:07:57.755 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:07:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:07:57.781 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:07:57.781 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-22T03:07:57.781 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:07:57.781 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:07:57.782 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:07:57.783 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:07:57.874 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:07:57.875 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:07:58.791 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.792 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.793 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-20.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:58.794 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:07:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:07:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-22T03:07:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-22T03:07:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-22T03:07:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-22T03:07:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-22T03:07:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:07:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-22T03:07:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-22T03:07:59.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-test 2024-09-22T03:07:59.741 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:27 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:07:59.873 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 124 M 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout: libasan x86_64 11.5.0-2.el9 appstream 414 k 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 158 k 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-22T03:07:59.874 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout:Install 7 Packages 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:07:59.875 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 125 M 2024-09-22T03:07:59.876 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 1.1 G 2024-09-22T03:07:59.876 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:08:00.048 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-22T03:08:00.049 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-22T03:08:00.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-20.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:00.051 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:08:00.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-test 2024-09-22T03:08:00.480 INFO:teuthology.orchestra.run.smithi146.stdout:(1/7): libasan-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 414 kB 00:00 2024-09-22T03:08:00.505 INFO:teuthology.orchestra.run.smithi146.stdout:(2/7): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 669 kB/s | 158 kB 00:00 2024-09-22T03:08:00.539 INFO:teuthology.orchestra.run.smithi146.stdout:(3/7): libubsan-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 183 kB 00:00 2024-09-22T03:08:00.589 INFO:teuthology.orchestra.run.smithi146.stdout:(4/7): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-22T03:08:00.631 INFO:teuthology.orchestra.run.smithi146.stdout:(5/7): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.9 MB/s | 122 kB 00:00 2024-09-22T03:08:00.706 INFO:teuthology.orchestra.run.smithi146.stdout:(6/7): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-22T03:08:00.963 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:28 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:08:01.095 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:08:01.095 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:08:01.095 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:08:01.095 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 124 M 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: libasan x86_64 11.5.0-2.el9 appstream 414 k 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 158 k 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:08:01.096 INFO:teuthology.orchestra.run.smithi148.stdout:Install 7 Packages 2024-09-22T03:08:01.097 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:01.097 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 125 M 2024-09-22T03:08:01.097 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 1.1 G 2024-09-22T03:08:01.097 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:08:01.803 INFO:teuthology.orchestra.run.smithi148.stdout:(1/7): libasan-11.5.0-2.el9.x86_64.rpm 1.8 MB/s | 414 kB 00:00 2024-09-22T03:08:01.887 INFO:teuthology.orchestra.run.smithi148.stdout:(2/7): libubsan-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 183 kB 00:00 2024-09-22T03:08:01.996 INFO:teuthology.orchestra.run.smithi148.stdout:(3/7): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-22T03:08:02.021 INFO:teuthology.orchestra.run.smithi148.stdout:(4/7): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 357 kB/s | 158 kB 00:00 2024-09-22T03:08:02.055 INFO:teuthology.orchestra.run.smithi148.stdout:(5/7): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-22T03:08:02.080 INFO:teuthology.orchestra.run.smithi148.stdout:(6/7): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.0 MB/s | 122 kB 00:00 2024-09-22T03:08:03.906 INFO:teuthology.orchestra.run.smithi146.stdout:(7/7): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 34 MB/s | 124 MB 00:03 2024-09-22T03:08:03.909 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:08:03.909 INFO:teuthology.orchestra.run.smithi146.stdout:Total 31 MB/s | 125 MB 00:04 2024-09-22T03:08:03.964 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:08:03.986 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:08:03.986 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:08:04.804 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:08:04.804 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:08:05.822 INFO:teuthology.orchestra.run.smithi148.stdout:(7/7): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 29 MB/s | 124 MB 00:04 2024-09-22T03:08:05.825 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:08:05.825 INFO:teuthology.orchestra.run.smithi148.stdout:Total 27 MB/s | 125 MB 00:04 2024-09-22T03:08:05.856 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:08:05.879 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:08:05.900 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:08:05.900 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:08:05.922 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/7 2024-09-22T03:08:05.967 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/7 2024-09-22T03:08:05.999 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/7 2024-09-22T03:08:06.041 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:08:06.090 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libasan-11.5.0-2.el9.x86_64 5/7 2024-09-22T03:08:06.118 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:08:06.721 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:08:06.721 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:08:07.790 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:08:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/7 2024-09-22T03:08:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/7 2024-09-22T03:08:07.953 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/7 2024-09-22T03:08:07.981 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:08:08.022 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libasan-11.5.0-2.el9.x86_64 5/7 2024-09-22T03:08:08.049 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:08:23.300 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:08:23.381 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:08:25.402 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:08:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:08:31.679 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:08:31.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/7 2024-09-22T03:08:31.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/7 2024-09-22T03:08:31.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libasan-11.5.0-2.el9.x86_64 3/7 2024-09-22T03:08:31.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:08:31.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 5/7 2024-09-22T03:08:31.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/7 2024-09-22T03:08:32.051 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 7/7 2024-09-22T03:08:32.051 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:08:32.051 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: libasan-11.5.0-2.el9.x86_64 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:08:32.052 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:08:32.366 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph 2024-09-22T03:08:32.950 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:00 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:08:33.084 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:08:33.086 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:08:33.086 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:08:33.086 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:08:33.086 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:08:33.086 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.3 M 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.9 M 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 120 M 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-22T03:08:33.087 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-22T03:08:33.088 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-22T03:08:33.089 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-22T03:08:33.090 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:08:33.090 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:08:33.090 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:08:33.090 INFO:teuthology.orchestra.run.smithi146.stdout:Install 38 Packages 2024-09-22T03:08:33.090 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:08:33.090 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 134 M 2024-09-22T03:08:33.091 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 1.4 G 2024-09-22T03:08:33.091 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:08:33.648 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:08:33.648 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/7 2024-09-22T03:08:33.648 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/7 2024-09-22T03:08:33.648 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libasan-11.5.0-2.el9.x86_64 3/7 2024-09-22T03:08:33.648 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:08:33.648 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 5/7 2024-09-22T03:08:33.648 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/7 2024-09-22T03:08:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 7/7 2024-09-22T03:08:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: libasan-11.5.0-2.el9.x86_64 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:34.047 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:08:34.382 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph 2024-09-22T03:08:34.974 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:02 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:08:35.108 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:08:35.110 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.3 M 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.9 M 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 120 M 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-22T03:08:35.111 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-22T03:08:35.112 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-22T03:08:35.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-22T03:08:35.114 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:35.114 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:08:35.114 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:08:35.114 INFO:teuthology.orchestra.run.smithi148.stdout:Install 38 Packages 2024-09-22T03:08:35.114 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:08:35.114 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 134 M 2024-09-22T03:08:35.114 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 1.4 G 2024-09-22T03:08:35.115 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:08:35.972 INFO:teuthology.orchestra.run.smithi148.stdout:(1/38): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 43 kB/s | 6.6 kB 00:00 2024-09-22T03:08:36.122 INFO:teuthology.orchestra.run.smithi148.stdout:(2/38): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 5.0 MB/s | 1.5 MB 00:00 2024-09-22T03:08:36.197 INFO:teuthology.orchestra.run.smithi148.stdout:(3/38): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 6.0 MB/s | 2.3 MB 00:00 2024-09-22T03:08:36.323 INFO:teuthology.orchestra.run.smithi148.stdout:(4/38): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-22T03:08:36.348 INFO:teuthology.orchestra.run.smithi148.stdout:(5/38): ceph-mgr-modules-core-19.1.1-373.g4e45e 1.6 MB/s | 249 kB 00:00 2024-09-22T03:08:36.490 INFO:teuthology.orchestra.run.smithi148.stdout:(6/38): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-22T03:08:36.540 INFO:teuthology.orchestra.run.smithi148.stdout:(7/38): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-22T03:08:36.590 INFO:teuthology.orchestra.run.smithi148.stdout:(8/38): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-09-22T03:08:36.640 INFO:teuthology.orchestra.run.smithi148.stdout:(9/38): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-22T03:08:36.707 INFO:teuthology.orchestra.run.smithi148.stdout:(10/38): python3-cryptography-36.0.1-4.el9.x86_ 3.4 MB/s | 1.2 MB 00:00 2024-09-22T03:08:36.732 INFO:teuthology.orchestra.run.smithi148.stdout:(11/38): python3-urllib3-1.26.5-6.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-09-22T03:08:36.861 INFO:teuthology.orchestra.run.smithi148.stdout:(12/38): protobuf-3.14.0-13.el9.x86_64.rpm 7.9 MB/s | 1.0 MB 00:00 2024-09-22T03:08:36.886 INFO:teuthology.orchestra.run.smithi148.stdout:(13/38): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-22T03:08:36.928 INFO:teuthology.orchestra.run.smithi148.stdout:(14/38): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-22T03:08:36.945 INFO:teuthology.orchestra.run.smithi148.stdout:(15/38): python3-markupsafe-1.1.1-12.el9.x86_64 592 kB/s | 35 kB 00:00 2024-09-22T03:08:36.995 INFO:teuthology.orchestra.run.smithi148.stdout:(16/38): python3-packaging-20.9-5.el9.noarch.rp 1.1 MB/s | 77 kB 00:00 2024-09-22T03:08:37.021 INFO:teuthology.orchestra.run.smithi148.stdout:(17/38): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-09-22T03:08:37.054 INFO:teuthology.orchestra.run.smithi148.stdout:(18/38): python3-toml-0.10.2-6.el9.noarch.rpm 708 kB/s | 42 kB 00:00 2024-09-22T03:08:37.082 INFO:teuthology.orchestra.run.smithi148.stdout:(19/38): lua-devel-5.4.4-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-09-22T03:08:37.130 INFO:teuthology.orchestra.run.smithi148.stdout:(20/38): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 868 kB/s | 43 kB 00:00 2024-09-22T03:08:37.164 INFO:teuthology.orchestra.run.smithi148.stdout:(21/38): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-09-22T03:08:37.197 INFO:teuthology.orchestra.run.smithi148.stdout:(22/38): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-22T03:08:37.223 INFO:teuthology.orchestra.run.smithi148.stdout:(23/38): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-22T03:08:37.248 INFO:teuthology.orchestra.run.smithi148.stdout:(24/38): python3-cherrypy-18.6.1-2.el9.noarch.r 4.2 MB/s | 358 kB 00:00 2024-09-22T03:08:37.273 INFO:teuthology.orchestra.run.smithi148.stdout:(25/38): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-09-22T03:08:37.298 INFO:teuthology.orchestra.run.smithi148.stdout:(26/38): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-22T03:08:37.323 INFO:teuthology.orchestra.run.smithi148.stdout:(27/38): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-22T03:08:37.349 INFO:teuthology.orchestra.run.smithi148.stdout:(28/38): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-22T03:08:37.374 INFO:teuthology.orchestra.run.smithi148.stdout:(29/38): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-22T03:08:37.399 INFO:teuthology.orchestra.run.smithi148.stdout:(30/38): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-22T03:08:37.425 INFO:teuthology.orchestra.run.smithi148.stdout:(31/38): python3-portend-3.1.0-2.el9.noarch.rpm 657 kB/s | 16 kB 00:00 2024-09-22T03:08:37.458 INFO:teuthology.orchestra.run.smithi148.stdout:(32/38): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-22T03:08:37.476 INFO:teuthology.orchestra.run.smithi148.stdout:(33/38): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-22T03:08:37.492 INFO:teuthology.orchestra.run.smithi148.stdout:(34/38): python3-tempora-5.0.0-2.el9.noarch.rpm 1.0 MB/s | 36 kB 00:00 2024-09-22T03:08:37.526 INFO:teuthology.orchestra.run.smithi148.stdout:(35/38): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-22T03:08:37.543 INFO:teuthology.orchestra.run.smithi148.stdout:(36/38): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-22T03:08:37.576 INFO:teuthology.orchestra.run.smithi148.stdout:(37/38): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-09-22T03:08:39.598 INFO:teuthology.orchestra.run.smithi146.stdout:(1/38): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 53 kB/s | 6.6 kB 00:00 2024-09-22T03:08:39.669 INFO:teuthology.orchestra.run.smithi148.stdout:(38/38): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_ 34 MB/s | 120 MB 00:03 2024-09-22T03:08:39.672 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:08:39.673 INFO:teuthology.orchestra.run.smithi148.stdout:Total 29 MB/s | 134 MB 00:04 2024-09-22T03:08:39.790 INFO:teuthology.orchestra.run.smithi146.stdout:(2/38): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 4.8 MB/s | 1.5 MB 00:00 2024-09-22T03:08:39.840 INFO:teuthology.orchestra.run.smithi146.stdout:(3/38): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 6.2 MB/s | 2.3 MB 00:00 2024-09-22T03:08:39.842 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:08:39.884 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:08:39.884 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:08:39.924 INFO:teuthology.orchestra.run.smithi146.stdout:(4/38): ceph-mgr-modules-core-19.1.1-373.g4e45e 2.9 MB/s | 249 kB 00:00 2024-09-22T03:08:39.999 INFO:teuthology.orchestra.run.smithi146.stdout:(5/38): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_6 12 MB/s | 4.9 MB 00:00 2024-09-22T03:08:40.116 INFO:teuthology.orchestra.run.smithi146.stdout:(6/38): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-22T03:08:40.158 INFO:teuthology.orchestra.run.smithi146.stdout:(7/38): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-22T03:08:40.200 INFO:teuthology.orchestra.run.smithi146.stdout:(8/38): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-22T03:08:40.225 INFO:teuthology.orchestra.run.smithi146.stdout:(9/38): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-22T03:08:40.254 INFO:teuthology.orchestra.run.smithi146.stdout:(10/38): python3-requests-2.25.1-8.el9.noarch.r 2.3 MB/s | 125 kB 00:00 2024-09-22T03:08:40.279 INFO:teuthology.orchestra.run.smithi146.stdout:(11/38): python3-urllib3-1.26.5-6.el9.noarch.rp 3.9 MB/s | 215 kB 00:00 2024-09-22T03:08:40.313 INFO:teuthology.orchestra.run.smithi146.stdout:(12/38): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-22T03:08:40.355 INFO:teuthology.orchestra.run.smithi146.stdout:(13/38): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-09-22T03:08:40.389 INFO:teuthology.orchestra.run.smithi146.stdout:(14/38): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-22T03:08:40.405 INFO:teuthology.orchestra.run.smithi146.stdout:(15/38): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-09-22T03:08:40.431 INFO:teuthology.orchestra.run.smithi146.stdout:(16/38): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-22T03:08:40.456 INFO:teuthology.orchestra.run.smithi146.stdout:(17/38): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-22T03:08:40.473 INFO:teuthology.orchestra.run.smithi146.stdout:(18/38): python3-toml-0.10.2-6.el9.noarch.rpm 995 kB/s | 42 kB 00:00 2024-09-22T03:08:40.532 INFO:teuthology.orchestra.run.smithi146.stdout:(19/38): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-22T03:08:40.565 INFO:teuthology.orchestra.run.smithi146.stdout:(20/38): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-09-22T03:08:40.582 INFO:teuthology.orchestra.run.smithi146.stdout:(21/38): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-22T03:08:40.624 INFO:teuthology.orchestra.run.smithi146.stdout:(22/38): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-22T03:08:40.649 INFO:teuthology.orchestra.run.smithi146.stdout:(23/38): python3-jaraco-8.2.1-3.el9.noarch.rpm 465 kB/s | 11 kB 00:00 2024-09-22T03:08:40.675 INFO:teuthology.orchestra.run.smithi146.stdout:(24/38): python3-jaraco-classes-3.2.1-5.el9.noa 711 kB/s | 18 kB 00:00 2024-09-22T03:08:40.700 INFO:teuthology.orchestra.run.smithi146.stdout:(25/38): python3-jaraco-collections-3.0.0-8.el9 937 kB/s | 23 kB 00:00 2024-09-22T03:08:40.725 INFO:teuthology.orchestra.run.smithi146.stdout:(26/38): python3-cherrypy-18.6.1-2.el9.noarch.r 2.5 MB/s | 358 kB 00:00 2024-09-22T03:08:40.750 INFO:teuthology.orchestra.run.smithi146.stdout:(27/38): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-22T03:08:40.776 INFO:teuthology.orchestra.run.smithi146.stdout:(28/38): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-22T03:08:40.801 INFO:teuthology.orchestra.run.smithi146.stdout:(29/38): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-22T03:08:40.826 INFO:teuthology.orchestra.run.smithi146.stdout:(30/38): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-22T03:08:40.851 INFO:teuthology.orchestra.run.smithi146.stdout:(31/38): python3-portend-3.1.0-2.el9.noarch.rpm 660 kB/s | 16 kB 00:00 2024-09-22T03:08:40.877 INFO:teuthology.orchestra.run.smithi146.stdout:(32/38): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-22T03:08:40.902 INFO:teuthology.orchestra.run.smithi146.stdout:(33/38): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-22T03:08:40.928 INFO:teuthology.orchestra.run.smithi146.stdout:(34/38): python3-tempora-5.0.0-2.el9.noarch.rpm 709 kB/s | 36 kB 00:00 2024-09-22T03:08:40.959 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:08:40.959 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:08:40.970 INFO:teuthology.orchestra.run.smithi146.stdout:(35/38): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-22T03:08:40.995 INFO:teuthology.orchestra.run.smithi146.stdout:(36/38): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-22T03:08:41.045 INFO:teuthology.orchestra.run.smithi146.stdout:(37/38): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-22T03:08:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:08:43.013 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/38 2024-09-22T03:08:43.052 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/38 2024-09-22T03:08:43.081 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/38 2024-09-22T03:08:43.152 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/38 2024-09-22T03:08:43.205 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/38 2024-09-22T03:08:43.238 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/38 2024-09-22T03:08:43.337 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/38 2024-09-22T03:08:43.407 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/38 2024-09-22T03:08:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/38 2024-09-22T03:08:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/38 2024-09-22T03:08:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/38 2024-09-22T03:08:43.548 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/38 2024-09-22T03:08:43.582 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/38 2024-09-22T03:08:43.622 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/38 2024-09-22T03:08:43.657 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/38 2024-09-22T03:08:43.691 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/38 2024-09-22T03:08:43.739 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/38 2024-09-22T03:08:43.772 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/38 2024-09-22T03:08:43.854 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/38 2024-09-22T03:08:44.000 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/38 2024-09-22T03:08:44.052 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/38 2024-09-22T03:08:44.620 INFO:teuthology.orchestra.run.smithi146.stdout:(38/38): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_ 25 MB/s | 120 MB 00:04 2024-09-22T03:08:44.625 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:08:44.625 INFO:teuthology.orchestra.run.smithi146.stdout:Total 12 MB/s | 134 MB 00:11 2024-09-22T03:08:44.796 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:08:44.836 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:08:44.836 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:08:45.914 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:08:45.914 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:08:47.915 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:08:47.981 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/38 2024-09-22T03:08:48.017 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/38 2024-09-22T03:08:48.050 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/38 2024-09-22T03:08:48.122 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/38 2024-09-22T03:08:48.157 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/38 2024-09-22T03:08:48.190 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/38 2024-09-22T03:08:48.291 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/38 2024-09-22T03:08:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/38 2024-09-22T03:08:48.425 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/38 2024-09-22T03:08:48.461 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/38 2024-09-22T03:08:48.499 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/38 2024-09-22T03:08:48.533 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/38 2024-09-22T03:08:48.567 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/38 2024-09-22T03:08:48.607 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/38 2024-09-22T03:08:48.642 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/38 2024-09-22T03:08:48.684 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/38 2024-09-22T03:08:48.732 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/38 2024-09-22T03:08:48.765 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/38 2024-09-22T03:08:48.837 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/38 2024-09-22T03:08:48.976 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/38 2024-09-22T03:08:49.028 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/38 2024-09-22T03:09:02.891 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/38 2024-09-22T03:09:02.924 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:09:02.953 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:09:02.953 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:02.953 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-22T03:09:02.953 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:09:02.953 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:09:02.953 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:03.099 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/38 2024-09-22T03:09:03.137 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/38 2024-09-22T03:09:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 26/38 2024-09-22T03:09:03.250 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 27/38 2024-09-22T03:09:03.436 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 28/38 2024-09-22T03:09:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 29/38 2024-09-22T03:09:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/38 2024-09-22T03:09:03.642 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 31/38 2024-09-22T03:09:03.682 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 32/38 2024-09-22T03:09:03.798 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 33/38 2024-09-22T03:09:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 34/38 2024-09-22T03:09:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:09:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:09:04.016 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:04.017 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-22T03:09:04.017 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:09:04.017 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:09:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:04.485 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:09:04.508 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:09:04.509 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:04.509 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-22T03:09:04.509 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:09:04.509 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:09:04.509 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:04.735 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:09:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:09:04.759 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:04.759 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-22T03:09:04.759 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:09:04.759 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:09:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:09:07.995 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/38 2024-09-22T03:09:08.024 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:09:08.053 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:09:08.053 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:08.053 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-22T03:09:08.053 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:09:08.053 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:09:08.053 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:08.166 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/38 2024-09-22T03:09:08.204 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/38 2024-09-22T03:09:08.253 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 26/38 2024-09-22T03:09:08.309 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 27/38 2024-09-22T03:09:08.496 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 28/38 2024-09-22T03:09:08.544 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 29/38 2024-09-22T03:09:08.619 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/38 2024-09-22T03:09:08.708 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 31/38 2024-09-22T03:09:08.741 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 32/38 2024-09-22T03:09:08.856 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 33/38 2024-09-22T03:09:09.018 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 34/38 2024-09-22T03:09:09.043 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:09:09.067 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:09:09.067 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:09.068 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-22T03:09:09.068 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:09:09.068 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:09:09.068 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:09.536 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:09:09.560 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:09:09.560 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:09.560 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-22T03:09:09.560 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:09:09.560 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:09:09.560 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:09.786 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:09:09.810 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:09:09.810 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:09.810 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-22T03:09:09.810 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:09:09.810 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:09:09.810 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:09.874 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/38 2024-09-22T03:09:15.041 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 14/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 15/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 16/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 17/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 18/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 19/38 2024-09-22T03:09:15.042 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 20/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 21/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 22/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 23/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 24/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 25/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 26/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 27/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 28/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 29/38 2024-09-22T03:09:15.043 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 30/38 2024-09-22T03:09:15.044 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 31/38 2024-09-22T03:09:15.044 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 32/38 2024-09-22T03:09:15.044 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 33/38 2024-09-22T03:09:15.045 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 34/38 2024-09-22T03:09:15.045 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 35/38 2024-09-22T03:09:15.045 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 36/38 2024-09-22T03:09:15.045 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 37/38 2024-09-22T03:09:16.173 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 38/38 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-22T03:09:16.174 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-22T03:09:16.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-22T03:09:16.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-22T03:09:16.177 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:16.177 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:16.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-base 2024-09-22T03:09:17.014 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:45 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:17.105 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:09:17.146 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:17.147 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:09:17.147 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:17.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install cephadm 2024-09-22T03:09:17.798 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:45 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:17.929 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:17.929 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:09:17.929 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:17.929 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-22T03:09:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:17.930 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:17.930 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:17.930 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:09:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:17.930 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 765 k 2024-09-22T03:09:17.931 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 771 k 2024-09-22T03:09:17.931 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:18.209 INFO:teuthology.orchestra.run.smithi148.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-22T03:09:18.209 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:18.210 INFO:teuthology.orchestra.run.smithi148.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-22T03:09:18.210 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:18.215 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:18.215 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:18.221 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:18.221 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:18.872 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:19.242 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:19.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:19.506 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:19.506 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:19.506 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:19.506 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:19.506 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:19.721 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-immutable-object-cache 2024-09-22T03:09:20.317 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:48 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:20.454 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:20.455 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:20.455 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-22T03:09:20.455 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:20.455 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:20.455 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 147 k 2024-09-22T03:09:20.455 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:20.455 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:20.456 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:20.456 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:09:20.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:20.456 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 147 k 2024-09-22T03:09:20.456 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 481 k 2024-09-22T03:09:20.456 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:20.492 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:09:20.492 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/38 2024-09-22T03:09:20.492 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/38 2024-09-22T03:09:20.492 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/38 2024-09-22T03:09:20.492 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/38 2024-09-22T03:09:20.492 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 14/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 15/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 16/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 17/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 18/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 19/38 2024-09-22T03:09:20.493 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 20/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 21/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 22/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 23/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 24/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 25/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 26/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 27/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 28/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 29/38 2024-09-22T03:09:20.494 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 30/38 2024-09-22T03:09:20.495 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 31/38 2024-09-22T03:09:20.495 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 32/38 2024-09-22T03:09:20.495 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 33/38 2024-09-22T03:09:20.495 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 34/38 2024-09-22T03:09:20.495 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 35/38 2024-09-22T03:09:20.495 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 36/38 2024-09-22T03:09:20.496 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 37/38 2024-09-22T03:09:20.658 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 727 kB/s | 147 kB 00:00 2024-09-22T03:09:20.659 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:20.659 INFO:teuthology.orchestra.run.smithi148.stdout:Total 722 kB/s | 147 kB 00:00 2024-09-22T03:09:20.659 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:20.667 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:20.667 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:20.725 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:20.726 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:20.864 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:20.926 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:09:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:09:20.952 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:20.952 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-22T03:09:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 38/38 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-22T03:09:21.397 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-22T03:09:21.398 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-22T03:09:21.399 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-22T03:09:21.400 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-22T03:09:21.400 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:21.400 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:21.663 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-base 2024-09-22T03:09:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:09:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:21.895 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:21.896 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:22.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr 2024-09-22T03:09:22.238 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:50 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:22.327 INFO:teuthology.orchestra.run.smithi146.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:09:22.369 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:22.370 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:09:22.370 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:22.437 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install cephadm 2024-09-22T03:09:22.717 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:50 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:22.807 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:09:22.851 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:22.852 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:09:22.852 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:22.925 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-dashboard 2024-09-22T03:09:23.018 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:51 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:23.150 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:09:23.151 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:23.152 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:09:23.152 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:23.152 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 765 k 2024-09-22T03:09:23.152 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 771 k 2024-09-22T03:09:23.152 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:09:23.426 INFO:teuthology.orchestra.run.smithi146.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-22T03:09:23.426 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:23.427 INFO:teuthology.orchestra.run.smithi146.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-22T03:09:23.427 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:09:23.432 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:09:23.432 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:09:23.437 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:09:23.438 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:09:23.498 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:51 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:23.511 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:09:23.630 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:23.631 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:09:23.631 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:23.631 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:23.632 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:23.633 INFO:teuthology.orchestra.run.smithi148.stdout:Install 11 Packages 2024-09-22T03:09:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:23.633 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 7.7 M 2024-09-22T03:09:23.633 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 101 M 2024-09-22T03:09:23.633 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:23.987 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:24.036 INFO:teuthology.orchestra.run.smithi148.stdout:(1/11): ceph-grafana-dashboards-19.1.1-373.g4e4 213 kB/s | 32 kB 00:00 2024-09-22T03:09:24.062 INFO:teuthology.orchestra.run.smithi148.stdout:(2/11): ceph-prometheus-alerts-19.1.1-373.g4e45 94 kB/s | 16 kB 00:00 2024-09-22T03:09:24.062 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:24.237 INFO:teuthology.orchestra.run.smithi148.stdout:(3/11): protobuf-compiler-3.14.0-13.el9.x86_64. 4.8 MB/s | 863 kB 00:00 2024-09-22T03:09:24.287 INFO:teuthology.orchestra.run.smithi148.stdout:(4/11): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 8.9 MB/s | 3.6 MB 00:00 2024-09-22T03:09:24.312 INFO:teuthology.orchestra.run.smithi148.stdout:(5/11): python3-protobuf-3.14.0-13.el9.noarch.r 977 kB/s | 269 kB 00:00 2024-09-22T03:09:24.354 INFO:teuthology.orchestra.run.smithi148.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-22T03:09:24.388 INFO:teuthology.orchestra.run.smithi148.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.5 MB/s | 144 kB 00:00 2024-09-22T03:09:24.395 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:24.422 INFO:teuthology.orchestra.run.smithi148.stdout:(8/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.9 MB/s | 548 kB 00:00 2024-09-22T03:09:24.439 INFO:teuthology.orchestra.run.smithi148.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 609 kB/s | 31 kB 00:00 2024-09-22T03:09:24.464 INFO:teuthology.orchestra.run.smithi148.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-22T03:09:24.522 INFO:teuthology.orchestra.run.smithi148.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2024-09-22T03:09:24.527 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:24.527 INFO:teuthology.orchestra.run.smithi148.stdout:Total 8.6 MB/s | 7.7 MB 00:00 2024-09-22T03:09:24.556 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:24.573 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:24.573 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:24.649 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:24.649 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:24.649 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:09:24.649 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:24.649 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:24.649 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:24.773 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:24.887 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-immutable-object-cache 2024-09-22T03:09:25.251 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:25.337 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-22T03:09:25.367 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-22T03:09:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-22T03:09:25.475 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:53 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:25.594 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-22T03:09:25.609 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:25.609 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:25.609 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-22T03:09:25.609 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:25.609 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:09:25.609 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 147 k 2024-09-22T03:09:25.609 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:25.610 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:09:25.610 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:25.610 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:09:25.610 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:25.610 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 147 k 2024-09-22T03:09:25.610 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 481 k 2024-09-22T03:09:25.610 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:09:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-22T03:09:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-22T03:09:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-22T03:09:25.814 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 724 kB/s | 147 kB 00:00 2024-09-22T03:09:25.815 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:25.815 INFO:teuthology.orchestra.run.smithi146.stdout:Total 718 kB/s | 147 kB 00:00 2024-09-22T03:09:25.815 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:09:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:09:25.822 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:09:25.823 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:09:25.849 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 9/11 2024-09-22T03:09:25.879 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:09:25.879 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:09:26.030 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:09:26.099 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:09:26.125 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:09:26.125 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:26.126 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-22T03:09:26.126 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:26.917 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 10/11 2024-09-22T03:09:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:09:27.139 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:09:27.139 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:27.139 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:09:27.139 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:27.139 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:27.139 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:27.367 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr 2024-09-22T03:09:27.965 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:55 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:28.056 INFO:teuthology.orchestra.run.smithi146.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:09:28.100 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:28.100 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:09:28.101 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:28.163 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-dashboard 2024-09-22T03:09:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-22T03:09:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-22T03:09:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-22T03:09:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:28.587 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:28.588 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:28.739 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:56 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:28.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-22T03:09:28.871 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:28.872 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:28.872 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:09:28.872 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:28.872 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:09:28.872 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:28.873 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:09:28.874 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:28.874 INFO:teuthology.orchestra.run.smithi146.stdout:Install 11 Packages 2024-09-22T03:09:28.874 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:28.874 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 7.7 M 2024-09-22T03:09:28.874 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 101 M 2024-09-22T03:09:28.874 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:09:29.417 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:57 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:29.550 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-22T03:09:29.551 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout:Install 12 Packages 2024-09-22T03:09:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:29.553 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 38 M 2024-09-22T03:09:29.553 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 212 M 2024-09-22T03:09:29.553 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:29.592 INFO:teuthology.orchestra.run.smithi146.stdout:(1/11): ceph-grafana-dashboards-19.1.1-373.g4e4 199 kB/s | 32 kB 00:00 2024-09-22T03:09:29.618 INFO:teuthology.orchestra.run.smithi146.stdout:(2/11): ceph-prometheus-alerts-19.1.1-373.g4e45 89 kB/s | 16 kB 00:00 2024-09-22T03:09:29.793 INFO:teuthology.orchestra.run.smithi146.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-22T03:09:29.843 INFO:teuthology.orchestra.run.smithi146.stdout:(4/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-22T03:09:29.845 INFO:teuthology.orchestra.run.smithi148.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-22T03:09:29.887 INFO:teuthology.orchestra.run.smithi148.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-22T03:09:29.910 INFO:teuthology.orchestra.run.smithi146.stdout:(5/11): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 7.5 MB/s | 3.6 MB 00:00 2024-09-22T03:09:29.935 INFO:teuthology.orchestra.run.smithi146.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 228 kB/s | 19 kB 00:00 2024-09-22T03:09:29.969 INFO:teuthology.orchestra.run.smithi146.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.4 MB/s | 144 kB 00:00 2024-09-22T03:09:29.971 INFO:teuthology.orchestra.run.smithi148.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-22T03:09:29.995 INFO:teuthology.orchestra.run.smithi146.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.3 MB/s | 31 kB 00:00 2024-09-22T03:09:30.013 INFO:teuthology.orchestra.run.smithi148.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 439 kB/s | 18 kB 00:00 2024-09-22T03:09:30.019 INFO:teuthology.orchestra.run.smithi146.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-22T03:09:30.063 INFO:teuthology.orchestra.run.smithi148.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-22T03:09:30.078 INFO:teuthology.orchestra.run.smithi146.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-22T03:09:30.197 INFO:teuthology.orchestra.run.smithi148.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-37 15 MB/s | 7.4 MB 00:00 2024-09-22T03:09:30.255 INFO:teuthology.orchestra.run.smithi148.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.1 MB/s | 3.0 MB 00:00 2024-09-22T03:09:30.381 INFO:teuthology.orchestra.run.smithi148.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-09-22T03:09:30.456 INFO:teuthology.orchestra.run.smithi148.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-22T03:09:30.481 INFO:teuthology.orchestra.run.smithi148.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-09-22T03:09:30.506 INFO:teuthology.orchestra.run.smithi148.stdout:(11/12): python3-devel-3.9.19-8.el9.x86_64.rpm 791 kB/s | 245 kB 00:00 2024-09-22T03:09:30.545 INFO:teuthology.orchestra.run.smithi146.stdout:(11/11): protobuf-compiler-3.14.0-13.el9.x86_64 931 kB/s | 863 kB 00:00 2024-09-22T03:09:30.551 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:30.551 INFO:teuthology.orchestra.run.smithi146.stdout:Total 4.6 MB/s | 7.7 MB 00:01 2024-09-22T03:09:30.581 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:09:30.599 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:09:30.599 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:09:30.774 INFO:teuthology.orchestra.run.smithi148.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2024-09-22T03:09:30.774 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:30.775 INFO:teuthology.orchestra.run.smithi148.stdout:Total 31 MB/s | 38 MB 00:01 2024-09-22T03:09:30.811 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:09:30.812 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:09:31.129 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:31.155 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:31.155 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:31.302 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:09:31.385 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-22T03:09:31.415 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-22T03:09:31.524 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-22T03:09:31.576 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:31.577 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:31.660 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-22T03:09:31.713 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-22T03:09:31.796 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-22T03:09:31.847 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-22T03:09:31.876 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:09:31.907 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 9/11 2024-09-22T03:09:32.356 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-22T03:09:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:09:32.572 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:09:32.614 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-22T03:09:32.801 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-22T03:09:32.829 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-22T03:09:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-22T03:09:32.967 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 10/11 2024-09-22T03:09:33.008 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:09:33.564 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-22T03:09:33.676 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-22T03:09:34.280 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-22T03:09:34.585 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-22T03:09:34.585 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-22T03:09:34.586 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:34.587 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:34.817 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-22T03:09:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:09:35.394 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:03 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:35.526 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:35.527 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-22T03:09:35.527 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:09:35.527 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-22T03:09:35.528 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout:Install 12 Packages 2024-09-22T03:09:35.529 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:35.530 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 38 M 2024-09-22T03:09:35.530 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 212 M 2024-09-22T03:09:35.530 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:09:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-22T03:09:35.970 INFO:teuthology.orchestra.run.smithi146.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-22T03:09:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:09:36.087 INFO:teuthology.orchestra.run.smithi146.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-09-22T03:09:36.120 INFO:teuthology.orchestra.run.smithi146.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-22T03:09:36.279 INFO:teuthology.orchestra.run.smithi146.stdout:(4/12): ceph-mgr-diskprediction-local-19.1.1-37 15 MB/s | 7.4 MB 00:00 2024-09-22T03:09:36.304 INFO:teuthology.orchestra.run.smithi146.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 99 kB/s | 18 kB 00:00 2024-09-22T03:09:36.363 INFO:teuthology.orchestra.run.smithi146.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 461 kB/s | 38 kB 00:00 2024-09-22T03:09:36.472 INFO:teuthology.orchestra.run.smithi146.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-22T03:09:36.763 INFO:teuthology.orchestra.run.smithi146.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.4 MB/s | 3.0 MB 00:00 2024-09-22T03:09:36.897 INFO:teuthology.orchestra.run.smithi146.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.8 MB/s | 250 kB 00:00 2024-09-22T03:09:38.139 INFO:teuthology.orchestra.run.smithi146.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.7 MB/s | 4.9 MB 00:01 2024-09-22T03:09:38.222 INFO:teuthology.orchestra.run.smithi146.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.9 MB/s | 5.1 MB 00:01 2024-09-22T03:09:38.582 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:09:38.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-22T03:09:38.582 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:09:38.583 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-22T03:09:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-22T03:09:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:38.954 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:38.955 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:39.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-rook 2024-09-22T03:09:39.647 INFO:teuthology.orchestra.run.smithi146.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.9 MB/s | 16 MB 00:02 2024-09-22T03:09:39.648 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:39.649 INFO:teuthology.orchestra.run.smithi146.stdout:Total 9.3 MB/s | 38 MB 00:04 2024-09-22T03:09:39.980 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:07 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:39.996 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:09:40.023 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:09:40.023 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:09:40.114 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:40.115 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:09:40.117 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:40.117 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-22T03:09:40.117 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:40.118 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:40.119 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:40.119 INFO:teuthology.orchestra.run.smithi148.stdout:Install 13 Packages 2024-09-22T03:09:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:40.119 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.2 M 2024-09-22T03:09:40.119 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 26 M 2024-09-22T03:09:40.119 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:40.439 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:09:40.439 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:09:40.504 INFO:teuthology.orchestra.run.smithi148.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 347 kB/s | 26 kB 00:00 2024-09-22T03:09:40.529 INFO:teuthology.orchestra.run.smithi148.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 194 kB/s | 19 kB 00:00 2024-09-22T03:09:40.587 INFO:teuthology.orchestra.run.smithi148.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-09-22T03:09:40.613 INFO:teuthology.orchestra.run.smithi148.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-22T03:09:40.646 INFO:teuthology.orchestra.run.smithi148.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.7 MB/s | 279 kB 00:00 2024-09-22T03:09:40.672 INFO:teuthology.orchestra.run.smithi148.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 919 kB/s | 54 kB 00:00 2024-09-22T03:09:40.697 INFO:teuthology.orchestra.run.smithi148.stdout:(7/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 184 kB/s | 49 kB 00:00 2024-09-22T03:09:40.747 INFO:teuthology.orchestra.run.smithi148.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 319 kB/s | 32 kB 00:00 2024-09-22T03:09:40.773 INFO:teuthology.orchestra.run.smithi148.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-22T03:09:40.798 INFO:teuthology.orchestra.run.smithi148.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-22T03:09:40.823 INFO:teuthology.orchestra.run.smithi148.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-22T03:09:40.849 INFO:teuthology.orchestra.run.smithi148.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-22T03:09:40.882 INFO:teuthology.orchestra.run.smithi148.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-22T03:09:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-09-22T03:09:40.951 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:40.966 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:40.966 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:41.057 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:41.058 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:41.199 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:09:41.263 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-22T03:09:41.303 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:09:41.403 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:09:41.438 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-22T03:09:41.555 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:41.622 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-22T03:09:41.654 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-22T03:09:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-22T03:09:41.730 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-22T03:09:41.767 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-22T03:09:41.822 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-22T03:09:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-22T03:09:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-22T03:09:42.003 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-22T03:09:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-22T03:09:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-22T03:09:42.381 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-22T03:09:42.500 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-22T03:09:42.884 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-22T03:09:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-22T03:09:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-22T03:09:43.023 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-22T03:09:43.053 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:09:43.741 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:09:43.869 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:09:43.869 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-22T03:09:43.869 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-22T03:09:43.869 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-22T03:09:43.870 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-22T03:09:44.202 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-22T03:09:44.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:44.202 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:44.202 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:44.203 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:44.472 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-cephadm 2024-09-22T03:09:44.769 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-22T03:09:44.794 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:09:45.058 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:13 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:45.192 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-22T03:09:45.193 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 7.0 M 2024-09-22T03:09:45.194 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 33 M 2024-09-22T03:09:45.195 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:45.509 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-22T03:09:45.559 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-22T03:09:45.584 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-22T03:09:45.609 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-22T03:09:45.709 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-22T03:09:45.851 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 311 kB/s | 146 kB 00:00 2024-09-22T03:09:45.853 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:45.853 INFO:teuthology.orchestra.run.smithi148.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-22T03:09:45.925 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:45.936 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:45.936 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:46.036 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:46.036 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:46.518 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-22T03:09:46.570 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-22T03:09:47.090 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-22T03:09:47.174 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-22T03:09:47.246 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-22T03:09:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-22T03:09:47.385 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-22T03:09:47.386 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:09:47.386 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-22T03:09:47.791 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-22T03:09:47.791 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:47.791 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:09:47.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-22T03:09:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-22T03:09:47.793 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:47.793 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:48.085 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:09:48.085 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-22T03:09:48.085 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-22T03:09:48.085 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-22T03:09:48.085 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-22T03:09:48.085 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-22T03:09:48.198 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-rook 2024-09-22T03:09:48.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:48.359 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:48.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-fuse 2024-09-22T03:09:48.795 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:16 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:48.929 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:48.930 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:48.930 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-22T03:09:48.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout:Install 13 Packages 2024-09-22T03:09:48.932 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:48.933 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 2.2 M 2024-09-22T03:09:48.933 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 26 M 2024-09-22T03:09:48.933 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:09:49.235 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:17 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:49.369 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 868 k 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-22T03:09:49.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:49.371 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:49.371 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:49.371 INFO:teuthology.orchestra.run.smithi148.stdout:Install 2 Packages 2024-09-22T03:09:49.371 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:49.371 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 948 k 2024-09-22T03:09:49.371 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 3.1 M 2024-09-22T03:09:49.371 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:49.553 INFO:teuthology.orchestra.run.smithi148.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 588 kB/s | 80 kB 00:00 2024-09-22T03:09:49.570 INFO:teuthology.orchestra.run.smithi146.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 229 kB/s | 26 kB 00:00 2024-09-22T03:09:49.586 INFO:teuthology.orchestra.run.smithi146.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 149 kB/s | 19 kB 00:00 2024-09-22T03:09:49.603 INFO:teuthology.orchestra.run.smithi146.stdout:(3/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 338 kB/s | 49 kB 00:00 2024-09-22T03:09:49.712 INFO:teuthology.orchestra.run.smithi148.stdout:(2/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 2.9 MB/s | 868 kB 00:00 2024-09-22T03:09:49.712 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:49.713 INFO:teuthology.orchestra.run.smithi148.stdout:Total 2.7 MB/s | 948 kB 00:00 2024-09-22T03:09:49.725 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:49.747 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:49.747 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:49.812 INFO:teuthology.orchestra.run.smithi146.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 705 kB/s | 159 kB 00:00 2024-09-22T03:09:49.828 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:49.828 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:49.871 INFO:teuthology.orchestra.run.smithi146.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 783 kB/s | 222 kB 00:00 2024-09-22T03:09:49.896 INFO:teuthology.orchestra.run.smithi146.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 641 kB/s | 54 kB 00:00 2024-09-22T03:09:49.946 INFO:teuthology.orchestra.run.smithi146.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 428 kB/s | 32 kB 00:00 2024-09-22T03:09:49.971 INFO:teuthology.orchestra.run.smithi146.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 187 kB/s | 14 kB 00:00 2024-09-22T03:09:49.996 INFO:teuthology.orchestra.run.smithi146.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.3 MB/s | 220 kB 00:00 2024-09-22T03:09:50.020 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:50.022 INFO:teuthology.orchestra.run.smithi146.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 666 kB/s | 279 kB 00:00 2024-09-22T03:09:50.047 INFO:teuthology.orchestra.run.smithi146.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-22T03:09:50.097 INFO:teuthology.orchestra.run.smithi146.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.1 MB/s | 1.0 MB 00:00 2024-09-22T03:09:50.122 INFO:teuthology.orchestra.run.smithi146.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 896 kB/s | 90 kB 00:00 2024-09-22T03:09:50.125 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:50.125 INFO:teuthology.orchestra.run.smithi146.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-09-22T03:09:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-22T03:09:50.191 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:09:50.206 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:09:50.207 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:09:50.243 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:09:50.294 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:09:50.299 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:09:50.807 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:09:50.898 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:09:50.898 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-22T03:09:50.981 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-22T03:09:51.041 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-22T03:09:51.096 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-22T03:09:51.133 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-22T03:09:51.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-22T03:09:51.163 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:51.163 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:51.163 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-22T03:09:51.163 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:51.163 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:51.165 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-22T03:09:51.269 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-22T03:09:51.378 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-22T03:09:51.443 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-22T03:09:51.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-volume 2024-09-22T03:09:52.061 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:20 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:52.169 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-22T03:09:52.200 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:52.200 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 285 k 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 285 k 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 1.3 M 2024-09-22T03:09:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:52.223 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-22T03:09:52.265 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-22T03:09:52.313 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-22T03:09:52.335 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:09:52.449 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-volume-19.1.1-373.g4e45e6b4.el9.noarch.rpm 1.1 MB/s | 285 kB 00:00 2024-09-22T03:09:52.449 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:52.450 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-22T03:09:52.450 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:52.458 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:52.458 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:52.492 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:52.492 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:52.776 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:52.792 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:52.792 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:09:52.792 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-22T03:09:52.792 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-22T03:09:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-22T03:09:53.096 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-22T03:09:53.096 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-22T03:09:53.096 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-22T03:09:53.096 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-22T03:09:53.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-22T03:09:53.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-22T03:09:53.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-22T03:09:53.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-22T03:09:53.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-22T03:09:53.433 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:53.433 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:53.695 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-cephadm 2024-09-22T03:09:53.759 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:09:53.759 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:53.759 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:53.760 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:53.760 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:54.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados-devel 2024-09-22T03:09:54.271 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:22 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:54.404 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:54.405 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:54.405 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:09:54.405 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:54.405 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout:Install 6 Packages 2024-09-22T03:09:54.406 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:54.407 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 7.0 M 2024-09-22T03:09:54.407 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 33 M 2024-09-22T03:09:54.407 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:09:54.671 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:22 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:09:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 127 k 2024-09-22T03:09:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 457 k 2024-09-22T03:09:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:55.007 INFO:teuthology.orchestra.run.smithi146.stdout:(1/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 687 kB/s | 146 kB 00:00 2024-09-22T03:09:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 624 kB/s | 127 kB 00:00 2024-09-22T03:09:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:Total 620 kB/s | 127 kB 00:00 2024-09-22T03:09:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:55.066 INFO:teuthology.orchestra.run.smithi146.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-22T03:09:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:55.091 INFO:teuthology.orchestra.run.smithi146.stdout:(3/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 837 kB/s | 248 kB 00:00 2024-09-22T03:09:55.116 INFO:teuthology.orchestra.run.smithi146.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-22T03:09:55.158 INFO:teuthology.orchestra.run.smithi146.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-22T03:09:55.234 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:55.317 INFO:teuthology.orchestra.run.smithi146.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-22T03:09:55.319 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:55.319 INFO:teuthology.orchestra.run.smithi146.stdout:Total 7.7 MB/s | 7.0 MB 00:00 2024-09-22T03:09:55.343 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:09:55.390 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:09:55.402 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:09:55.402 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:09:55.501 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:09:55.502 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:09:55.751 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:09:55.836 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:09:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:09:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:56.008 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:56.009 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:56.016 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-22T03:09:56.068 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-22T03:09:56.237 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs2 2024-09-22T03:09:56.590 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-22T03:09:56.679 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-22T03:09:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-22T03:09:56.772 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:09:56.826 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:24 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:56.916 INFO:teuthology.orchestra.run.smithi148.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:09:56.960 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:56.961 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:09:56.961 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:57.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs-devel 2024-09-22T03:09:57.522 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:09:57.522 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-22T03:09:57.522 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-22T03:09:57.523 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-22T03:09:57.523 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-22T03:09:57.523 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-22T03:09:57.626 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:25 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:57.759 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:57.759 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:09:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:57.761 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 32 k 2024-09-22T03:09:57.761 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 142 k 2024-09-22T03:09:57.761 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:09:57.789 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-22T03:09:57.789 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:57.789 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:09:57.789 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:09:57.789 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-22T03:09:57.789 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-22T03:09:57.790 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-22T03:09:57.790 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-22T03:09:57.790 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-22T03:09:57.790 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:57.790 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:09:57.915 INFO:teuthology.orchestra.run.smithi148.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 205 kB/s | 32 kB 00:00 2024-09-22T03:09:57.916 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:57.916 INFO:teuthology.orchestra.run.smithi148.stdout:Total 203 kB/s | 32 kB 00:00 2024-09-22T03:09:57.916 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:09:57.920 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:09:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:09:57.933 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:09:57.933 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:09:58.024 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:09:58.047 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-fuse 2024-09-22T03:09:58.138 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:09:58.503 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:09:58.630 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:26 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:09:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:58.740 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:09:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:09:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:09:58.741 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:58.764 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 868 k 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:09:58.765 INFO:teuthology.orchestra.run.smithi146.stdout:Install 2 Packages 2024-09-22T03:09:58.766 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:09:58.766 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 948 k 2024-09-22T03:09:58.766 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 3.1 M 2024-09-22T03:09:58.766 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:09:58.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados2 2024-09-22T03:09:59.051 INFO:teuthology.orchestra.run.smithi146.stdout:(1/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 3.3 MB/s | 868 kB 00:00 2024-09-22T03:09:59.426 INFO:teuthology.orchestra.run.smithi146.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 126 kB/s | 80 kB 00:00 2024-09-22T03:09:59.427 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:09:59.427 INFO:teuthology.orchestra.run.smithi146.stdout:Total 1.4 MB/s | 948 kB 00:00 2024-09-22T03:09:59.440 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:09:59.462 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:09:59.462 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:09:59.501 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:27 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:09:59.543 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:09:59.544 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:09:59.592 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:09:59.636 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:09:59.637 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:09:59.637 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:09:59.711 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librbd1 2024-09-22T03:09:59.726 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:09:59.854 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-22T03:09:59.925 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:10:00.321 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:28 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:00.422 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:00.467 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:00.468 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:10:00.468 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:00.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rados 2024-09-22T03:10:00.629 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:10:00.630 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-22T03:10:00.887 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-22T03:10:00.887 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:00.887 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:10:00.887 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-22T03:10:00.888 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:00.888 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:01.120 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:29 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:01.164 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-volume 2024-09-22T03:10:01.210 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:01.255 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:01.256 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:10:01.256 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:01.326 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rgw 2024-09-22T03:10:01.764 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:29 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:01.897 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:01.898 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:01.898 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-22T03:10:01.898 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:01.898 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:10:01.898 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 285 k 2024-09-22T03:10:01.898 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:01.898 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:10:01.899 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:01.899 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:10:01.899 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:01.899 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 285 k 2024-09-22T03:10:01.899 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 1.3 M 2024-09-22T03:10:01.899 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:10:01.902 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:29 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:01.992 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:02.035 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:02.036 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:10:02.037 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:02.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-cephfs 2024-09-22T03:10:02.114 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-volume-19.1.1-373.g4e45e6b4.el9.noarch.rpm 1.3 MB/s | 285 kB 00:00 2024-09-22T03:10:02.114 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:02.115 INFO:teuthology.orchestra.run.smithi146.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-22T03:10:02.115 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:10:02.123 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:10:02.123 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:10:02.152 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:10:02.153 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:10:02.368 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:10:02.433 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:10:02.449 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:10:02.449 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:10:02.449 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-22T03:10:02.449 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:02.686 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:30 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:02.774 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:02.817 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:02.818 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:10:02.818 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:02.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rbd 2024-09-22T03:10:03.478 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:31 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:03.509 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:10:03.509 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:03.509 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:10:03.509 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:10:03.509 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:03.510 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:03.568 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:03.612 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:03.613 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-22T03:10:03.613 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:03.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-fuse 2024-09-22T03:10:03.839 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install librados-devel 2024-09-22T03:10:04.280 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:32 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:04.415 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 85 k 2024-09-22T03:10:04.417 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 256 k 2024-09-22T03:10:04.417 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:10:04.432 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:32 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:04.568 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:04.568 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:04.568 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-22T03:10:04.568 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:04.568 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 127 k 2024-09-22T03:10:04.569 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 457 k 2024-09-22T03:10:04.570 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:10:04.610 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 442 kB/s | 85 kB 00:00 2024-09-22T03:10:04.610 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:04.611 INFO:teuthology.orchestra.run.smithi148.stdout:Total 439 kB/s | 85 kB 00:00 2024-09-22T03:10:04.611 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:10:04.618 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:10:04.618 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:10:04.681 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:10:04.682 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:10:04.777 INFO:teuthology.orchestra.run.smithi146.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 610 kB/s | 127 kB 00:00 2024-09-22T03:10:04.778 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:04.778 INFO:teuthology.orchestra.run.smithi146.stdout:Total 606 kB/s | 127 kB 00:00 2024-09-22T03:10:04.778 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:10:04.784 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:10:04.784 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:10:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:10:04.847 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:10:04.848 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:10:04.949 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:10:05.118 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:05.562 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:05.625 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:05.626 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:10:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:05.626 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:05.801 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:05.801 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:05.802 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:10:05.802 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:05.802 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:05.802 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:05.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-mirror 2024-09-22T03:10:06.064 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install libcephfs2 2024-09-22T03:10:06.537 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:34 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:06.660 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:34 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:06.681 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:06.681 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:10:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:06.683 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.1 M 2024-09-22T03:10:06.683 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 14 M 2024-09-22T03:10:06.683 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:10:06.752 INFO:teuthology.orchestra.run.smithi146.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:06.797 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:06.799 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:10:06.799 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:06.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install libcephfs-devel 2024-09-22T03:10:07.014 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 9.2 MB/s | 3.1 MB 00:00 2024-09-22T03:10:07.014 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:07.015 INFO:teuthology.orchestra.run.smithi148.stdout:Total 9.2 MB/s | 3.1 MB 00:00 2024-09-22T03:10:07.015 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:10:07.024 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:10:07.024 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:10:07.110 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:10:07.110 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:10:07.445 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:35 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:07.563 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:10:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:07.577 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-22T03:10:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 32 k 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 142 k 2024-09-22T03:10:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:10:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:07.683 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:10:07.683 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-22T03:10:07.683 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:10:07.683 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:10:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 197 kB/s | 32 kB 00:00 2024-09-22T03:10:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:Total 195 kB/s | 32 kB 00:00 2024-09-22T03:10:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:10:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:10:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:10:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:10:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:10:07.849 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:10:07.953 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:08.328 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:08.566 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:08.566 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:08.566 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:10:08.567 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:08.567 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:08.567 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:08.738 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install librados2 2024-09-22T03:10:08.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:08.797 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:08.797 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:10:08.797 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:08.798 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:08.798 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:09.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-nbd 2024-09-22T03:10:09.322 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:37 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:09.414 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:09.458 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:09.459 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:10:09.459 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:09.525 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install librbd1 2024-09-22T03:10:09.703 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:37 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:09.839 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 169 k 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 169 k 2024-09-22T03:10:09.841 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 540 k 2024-09-22T03:10:09.841 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-22T03:10:10.042 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 843 kB/s | 169 kB 00:00 2024-09-22T03:10:10.043 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:10.043 INFO:teuthology.orchestra.run.smithi148.stdout:Total 836 kB/s | 169 kB 00:00 2024-09-22T03:10:10.043 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-22T03:10:10.051 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-22T03:10:10.051 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-22T03:10:10.111 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:38 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:10.115 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-22T03:10:10.116 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-22T03:10:10.202 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:10.246 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:10.248 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:10:10.248 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-22T03:10:10.309 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-rados 2024-09-22T03:10:10.381 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:10.893 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:38 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:10.981 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:11.025 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:11.026 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:10:11.026 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:11.091 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-rgw 2024-09-22T03:10:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:11.191 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-22T03:10:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:11.191 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-22T03:10:11.482 DEBUG:teuthology.parallel:result is None 2024-09-22T03:10:11.666 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:39 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:11.756 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:11.800 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:11.801 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:10:11.801 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:11.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-cephfs 2024-09-22T03:10:12.445 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:40 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:12.534 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:12.579 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:12.580 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:10:12.580 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:12.641 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-rbd 2024-09-22T03:10:13.215 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:41 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:13.305 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:10:13.348 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:13.349 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-22T03:10:13.349 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:13.412 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install rbd-fuse 2024-09-22T03:10:13.993 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:41 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:14.129 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:14.130 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:14.130 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-22T03:10:14.130 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:14.130 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:10:14.130 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 85 k 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 256 k 2024-09-22T03:10:14.131 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:10:14.500 INFO:teuthology.orchestra.run.smithi146.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 231 kB/s | 85 kB 00:00 2024-09-22T03:10:14.501 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:14.501 INFO:teuthology.orchestra.run.smithi146.stdout:Total 231 kB/s | 85 kB 00:00 2024-09-22T03:10:14.502 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:10:14.508 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:10:14.508 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:10:14.571 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:10:14.572 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:10:14.718 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:10:14.823 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:15.215 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:15.458 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:15.458 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:15.458 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:10:15.459 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:15.459 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:15.459 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:15.712 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install rbd-mirror 2024-09-22T03:10:16.326 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:44 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:16.468 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:16.468 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-22T03:10:16.468 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:16.468 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:10:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:10:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:16.469 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:10:16.469 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:16.469 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:10:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:16.469 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 3.1 M 2024-09-22T03:10:16.469 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 14 M 2024-09-22T03:10:16.470 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:10:17.295 INFO:teuthology.orchestra.run.smithi146.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 3.7 MB/s | 3.1 MB 00:00 2024-09-22T03:10:17.296 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:17.296 INFO:teuthology.orchestra.run.smithi146.stdout:Total 3.7 MB/s | 3.1 MB 00:00 2024-09-22T03:10:17.297 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:10:17.306 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:10:17.306 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:10:17.387 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:10:17.387 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:10:17.844 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:10:17.904 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:17.930 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:17.930 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:10:17.931 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-22T03:10:17.931 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:10:17.931 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:10:17.931 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:19.012 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:19.012 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:19.012 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:10:19.012 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:19.012 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:19.012 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:19.247 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install rbd-nbd 2024-09-22T03:10:19.854 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:02:47 ago on Sun 22 Sep 2024 03:07:32 AM UTC. 2024-09-22T03:10:19.993 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 169 k 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-22T03:10:19.994 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:19.995 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 169 k 2024-09-22T03:10:19.995 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 540 k 2024-09-22T03:10:19.995 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-22T03:10:20.282 INFO:teuthology.orchestra.run.smithi146.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 590 kB/s | 169 kB 00:00 2024-09-22T03:10:20.283 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:10:20.283 INFO:teuthology.orchestra.run.smithi146.stdout:Total 587 kB/s | 169 kB 00:00 2024-09-22T03:10:20.283 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-22T03:10:20.291 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-22T03:10:20.291 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-22T03:10:20.356 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-22T03:10:20.356 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-22T03:10:20.520 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-22T03:10:20.612 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:21.024 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:21.277 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:10:21.277 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:21.277 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-22T03:10:21.277 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:10:21.277 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:21.277 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-22T03:10:21.537 DEBUG:teuthology.parallel:result is None 2024-09-22T03:10:21.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:10:21.682 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-22T03:10:21.704 INFO:teuthology.orchestra.run.smithi146.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-22T03:10:21.704 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-22T03:10:21.704 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-22T03:10:21.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:10:21.848 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-22T03:10:21.874 INFO:teuthology.orchestra.run.smithi148.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-22T03:10:21.874 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-22T03:10:21.875 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-22T03:10:21.876 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-22T03:10:21.876 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:21.877 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T03:10:21.902 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:21.903 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T03:10:21.945 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-22T03:10:21.945 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:21.945 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T03:10:21.969 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T03:10:22.035 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:22.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T03:10:22.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T03:10:22.132 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-22T03:10:22.132 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:22.132 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T03:10:22.157 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T03:10:22.221 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:22.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T03:10:22.247 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T03:10:22.313 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-22T03:10:22.313 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:22.313 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T03:10:22.338 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T03:10:22.403 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:22.403 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T03:10:22.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T03:10:22.493 INFO:teuthology.run_tasks:Running task ceph... 2024-09-22T03:10:22.569 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-22T03:10:22.569 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2024-09-22T03:10:22.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-09-22T03:10:22.597 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-22T03:10:22.597 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T03:10:22.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T03:10:22.662 INFO:tasks.ceph:Creating extra log directories... 2024-09-22T03:10:22.662 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T03:10:22.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T03:10:22.737 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-22T03:10:22.737 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-22T03:10:22.737 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732', 'branch': 'squid', 'description': 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914732', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'squid', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1158, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'crimson-rados', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi146.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJQlVPTXtg49Z8Acf51W4FW0JfXcXg7psEnOFhZars3BzTz/AMDpjoCVvpyMe3/DKrZLBFzEk7f/bjjJRiTdZqk=', 'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCiwpGJfMU0EBteNF0PggHbh+k1vMXFqSA+IdXAHwMhJLg9EYKEozrW3AP2xq0JknUfFzJSCRj/9FNbc2EofzXg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'full_sequential': [{'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}]}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:56:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-22T03:10:22.737 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T03:10:22.768 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T03:10:22.783 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T03:10:22.823 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T03:10:22.848 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:22.848 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T03:10:22.901 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T03:10:22.901 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-22 03:10:18.188627398 +0000 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-22 03:06:52.156711190 +0000 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-22 03:06:52.156711190 +0000 2024-09-22T03:10:22.957 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-22 03:06:52.156711190 +0000 2024-09-22T03:10:22.958 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T03:10:23.021 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-22T03:10:23.021 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-22T03:10:23.021 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000206059 s, 2.5 MB/s 2024-09-22T03:10:23.022 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T03:10:23.079 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-22 03:10:18.189627383 +0000 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-22 03:06:52.479706281 +0000 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-22 03:06:52.479706281 +0000 2024-09-22T03:10:23.134 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-22 03:06:52.479706281 +0000 2024-09-22T03:10:23.135 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T03:10:23.198 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-22T03:10:23.198 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-22T03:10:23.198 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000195886 s, 2.6 MB/s 2024-09-22T03:10:23.199 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T03:10:23.255 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-09-22T03:10:23.310 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-22 03:10:18.189627383 +0000 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-22 03:06:52.804701342 +0000 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-22 03:06:52.805701327 +0000 2024-09-22T03:10:23.311 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-22 03:06:52.804701342 +0000 2024-09-22T03:10:23.311 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T03:10:23.373 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-22T03:10:23.373 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-22T03:10:23.373 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000191628 s, 2.7 MB/s 2024-09-22T03:10:23.374 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T03:10:23.431 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-09-22T03:10:23.488 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T03:10:23.489 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:23.489 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-22T03:10:23.489 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:23.489 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:23.489 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-22 03:10:18.189627383 +0000 2024-09-22T03:10:23.489 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-22 03:06:53.124696479 +0000 2024-09-22T03:10:23.489 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-22 03:06:53.124696479 +0000 2024-09-22T03:10:23.490 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-22 03:06:53.124696479 +0000 2024-09-22T03:10:23.490 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T03:10:23.551 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-22T03:10:23.552 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-22T03:10:23.552 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000198825 s, 2.6 MB/s 2024-09-22T03:10:23.553 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T03:10:23.609 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:10:23.609 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:23.609 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T03:10:23.627 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T03:10:23.627 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-09-22T03:10:23.684 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-22 03:10:07.942345565 +0000 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-22 03:06:52.861175170 +0000 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-22 03:06:52.861175170 +0000 2024-09-22T03:10:23.685 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-22 03:06:52.861175170 +0000 2024-09-22T03:10:23.686 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T03:10:23.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-22T03:10:23.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-22T03:10:23.750 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000293505 s, 1.7 MB/s 2024-09-22T03:10:23.751 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T03:10:23.809 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-09-22T03:10:23.865 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T03:10:23.865 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:23.865 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-22T03:10:23.865 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:23.865 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:23.866 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-22 03:10:07.942345565 +0000 2024-09-22T03:10:23.866 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-22 03:06:53.201167056 +0000 2024-09-22T03:10:23.866 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-22 03:06:53.201167056 +0000 2024-09-22T03:10:23.866 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-22 03:06:53.201167056 +0000 2024-09-22T03:10:23.866 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T03:10:23.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-22T03:10:23.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-22T03:10:23.930 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.0002087 s, 2.5 MB/s 2024-09-22T03:10:23.931 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T03:10:23.988 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-09-22T03:10:24.044 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T03:10:24.045 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:24.045 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-22T03:10:24.045 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:24.045 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:24.045 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-22 03:10:07.942345565 +0000 2024-09-22T03:10:24.045 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-22 03:06:53.548158774 +0000 2024-09-22T03:10:24.045 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-22 03:06:53.548158774 +0000 2024-09-22T03:10:24.046 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-22 03:06:53.548158774 +0000 2024-09-22T03:10:24.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T03:10:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-22T03:10:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-22T03:10:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000215514 s, 2.4 MB/s 2024-09-22T03:10:24.110 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T03:10:24.167 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-09-22T03:10:24.221 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T03:10:24.221 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:10:24.222 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-22T03:10:24.222 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:10:24.222 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:10:24.222 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-22 03:10:07.943345541 +0000 2024-09-22T03:10:24.222 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-22 03:06:53.864151232 +0000 2024-09-22T03:10:24.222 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-22 03:06:53.864151232 +0000 2024-09-22T03:10:24.222 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-22 03:06:53.864151232 +0000 2024-09-22T03:10:24.222 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T03:10:24.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-22T03:10:24.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-22T03:10:24.285 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000247518 s, 2.1 MB/s 2024-09-22T03:10:24.286 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T03:10:24.342 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:10:24.343 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-22T03:10:24.343 INFO:tasks.ceph:Generating config... 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] bluestore rocksdb cf = False 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-22T03:10:24.344 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] err to stderr = True 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] log flush on exit = True 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] log to file = False 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] log to stderr = True 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] osd delete sleep = 1 2024-09-22T03:10:24.345 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-22T03:10:24.346 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-22T03:10:24.346 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-22T03:10:24.346 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-22T03:10:24.346 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-22T03:10:24.346 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-22T03:10:24.346 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-22T03:10:24.346 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2024-09-22T03:10:24.346 INFO:tasks.ceph:[client.0] debug ms = 1 2024-09-22T03:10:24.346 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-09-22T03:10:24.346 INFO:tasks.ceph:[client.0] debug rados = 20 2024-09-22T03:10:24.346 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:10:24.346 INFO:tasks.ceph:[global] ms client mode = crc 2024-09-22T03:10:24.346 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-09-22T03:10:24.347 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-09-22T03:10:24.347 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-09-22T03:10:24.347 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-09-22T03:10:24.347 INFO:tasks.ceph:[global] ms service mode = crc 2024-09-22T03:10:24.347 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2024-09-22T03:10:24.347 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-22T03:10:24.347 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-22T03:10:24.348 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-22T03:10:24.348 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-09-22T03:10:24.348 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-22T03:10:24.348 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-09-22T03:10:24.348 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-09-22T03:10:24.348 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-09-22T03:10:24.348 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-09-22T03:10:24.348 INFO:tasks.ceph:Setting up mon.a... 2024-09-22T03:10:24.348 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-22T03:10:24.402 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.keyring 2024-09-22T03:10:24.405 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-22T03:10:24.454 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:10:24.478 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.146')] 2024-09-22T03:10:24.479 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.146', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-09-22T03:10:24.479 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:24.480 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T03:10:24.534 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.146 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:24.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:24.618+0000 7fda1228ae80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:24.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:24.618+0000 7fda1228ae80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:24.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:24.618+0000 7fda1228ae80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: generated fsid 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:setting min_mon_release = quincy 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:epoch 0 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:fsid 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:last_changed 2024-09-22T03:10:24.619690+0000 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:created 2024-09-22T03:10:24.619690+0000 2024-09-22T03:10:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:min_mon_release 17 (quincy) 2024-09-22T03:10:24.622 INFO:teuthology.orchestra.run.smithi146.stdout:election_strategy: 1 2024-09-22T03:10:24.622 INFO:teuthology.orchestra.run.smithi146.stdout:0: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.a 2024-09-22T03:10:24.622 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-22T03:10:24.623 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T03:10:24.638 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 39bf19db-1c34-4685-8c41-ee47ed066388... 2024-09-22T03:10:24.639 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T03:10:24.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-22T03:10:24.722 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-22T03:10:24.723 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.724 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = 172.21.15.146 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: ms client mode = crc 2024-09-22T03:10:24.725 INFO:teuthology.orchestra.run.smithi146.stdout: ms cluster mode = crc 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: ms mon client mode = crc 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: ms mon cluster mode = crc 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: ms mon service mode = crc 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: ms service mode = crc 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_min_size = 2 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_size = 3 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 120 2024-09-22T03:10:24.726 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.727 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: debug monc = 20 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-22T03:10:24.728 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore compression algorithm = snappy 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore compression mode = aggressive 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore rocksdb cf = False 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: crimson alien thread cpu cores = 6-7 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: crimson osd obc lru size = 10 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: err to stderr = True 2024-09-22T03:10:24.729 INFO:teuthology.orchestra.run.smithi146.stdout: log flush on exit = True 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: log to file = False 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: log to stderr = True 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd blocked scrub grace period = 3600 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug reject backfill probability = 0.3 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd delete sleep = 1 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd max backfills = 3 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd max markdown count = 1000 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub min interval = 60 2024-09-22T03:10:24.730 INFO:teuthology.orchestra.run.smithi146.stdout: osd shutdown pgref assert = True 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: osd snap trim sleep = 2 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:10:24.731 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-22T03:10:24.732 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-22T03:10:24.733 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: mon min osdmap epochs = 50 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: mon osdmap full prune interval = 2 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: mon osdmap full prune min = 15 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: mon osdmap full prune txsize = 2 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default crimson = True 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: paxos service trim min = 10 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T03:10:24.734 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout: debug objecter = 20 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout: debug rados = 20 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2024-09-22T03:10:24.735 INFO:teuthology.orchestra.run.smithi146.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:10:24.736 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2024-09-22T03:10:24.736 INFO:teuthology.orchestra.run.smithi146.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:10:24.737 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-22T03:10:24.737 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-09-22T03:10:24.737 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T03:10:24.737 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-09-22T03:10:24.737 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.737 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-09-22T03:10:24.737 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.738 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T03:10:24.739 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = 172.21.15.146 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: ms client mode = crc 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: ms cluster mode = crc 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: ms mon client mode = crc 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: ms mon cluster mode = crc 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: ms mon service mode = crc 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: ms service mode = crc 2024-09-22T03:10:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_min_size = 2 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_size = 3 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 120 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.741 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.742 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: debug monc = 20 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore compression algorithm = snappy 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore compression mode = aggressive 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore rocksdb cf = False 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: crimson alien thread cpu cores = 6-7 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: crimson osd obc lru size = 10 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-09-22T03:10:24.743 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: err to stderr = True 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: log flush on exit = True 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: log to file = False 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: log to stderr = True 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: osd blocked scrub grace period = 3600 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug reject backfill probability = 0.3 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: osd delete sleep = 1 2024-09-22T03:10:24.744 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: osd max backfills = 3 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: osd max markdown count = 1000 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub min interval = 60 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: osd shutdown pgref assert = True 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: osd snap trim sleep = 2 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:10:24.745 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.746 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon min osdmap epochs = 50 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon osdmap full prune interval = 2 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon osdmap full prune min = 15 2024-09-22T03:10:24.747 INFO:teuthology.orchestra.run.smithi148.stdout: mon osdmap full prune txsize = 2 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default crimson = True 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: paxos service trim min = 10 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: debug objecter = 20 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout: debug rados = 20 2024-09-22T03:10:24.748 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-09-22T03:10:24.749 INFO:teuthology.orchestra.run.smithi148.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:10:24.749 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-09-22T03:10:24.749 INFO:teuthology.orchestra.run.smithi148.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:10:24.749 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-09-22T03:10:24.749 INFO:teuthology.orchestra.run.smithi148.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:10:24.749 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2024-09-22T03:10:24.749 INFO:teuthology.orchestra.run.smithi148.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:10:24.749 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-22T03:10:24.749 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-22T03:10:24.838 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-22T03:10:24.839 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:24.839 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-22T03:10:24.854 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:24.854 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-22T03:10:24.909 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2024-09-22T03:10:24.909 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:24.909 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T03:10:24.910 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:10:24.984 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:24.984 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:25.040 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-09-22T03:10:25.040 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:25.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T03:10:25.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:10:25.076 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:25.076 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:25.131 INFO:tasks.ceph:Setting up mon nodes... 2024-09-22T03:10:25.131 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-22T03:10:25.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T03:10:25.239 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T03:10:25.242 INFO:tasks.ceph:Setting up mds nodes... 2024-09-22T03:10:25.242 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-22T03:10:25.242 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-22T03:10:25.289 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-22T03:10:25.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-22T03:10:25.349 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-22T03:10:25.361 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-22T03:10:25.362 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-22T03:10:25.362 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-22T03:10:25.387 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:10:25.387 INFO:tasks.ceph:role: osd.0 2024-09-22T03:10:25.387 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-22T03:10:25.387 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T03:10:25.459 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:10:25.460 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:10:25.980 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-09-22T03:10:25.982 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-22T03:10:25.982 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-22T03:10:26.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-22T03:10:26.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-22T03:10:26.207 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:10:26.207 INFO:tasks.ceph:role: osd.1 2024-09-22T03:10:26.207 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-22T03:10:26.207 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T03:10:26.278 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:10:26.278 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:10:26.279 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:10:26.742 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-09-22T03:10:26.744 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-22T03:10:26.744 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-22T03:10:26.838 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-22T03:10:26.867 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:27.420 INFO:teuthology.orchestra.run.smithi146.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:27.421 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:27,420 seastar - Reactor backend: linux-aio 2024-09-22T03:10:27.443 INFO:teuthology.orchestra.run.smithi146.stderr:==27718==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:27.488 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:27,487 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:10:27.488 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:27,487 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-22T03:10:27.489 INFO:teuthology.orchestra.run.smithi146.stderr:DEBUG 2024-09-22 03:10:27,488 [shard 0:main] seastar - reactor::drain 2024-09-22T03:10:28.006 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:28.007 INFO:teuthology.orchestra.run.smithi146.stderr:Suppressions used: 2024-09-22T03:10:28.007 INFO:teuthology.orchestra.run.smithi146.stderr: count bytes template 2024-09-22T03:10:28.007 INFO:teuthology.orchestra.run.smithi146.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:28.007 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:28.007 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:10:28.065 INFO:teuthology.orchestra.run.smithi146.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:28.066 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:28,066 seastar - Reactor backend: linux-aio 2024-09-22T03:10:28.102 INFO:teuthology.orchestra.run.smithi146.stderr:==27716==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:33.183 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:33.184 INFO:teuthology.orchestra.run.smithi146.stderr:Suppressions used: 2024-09-22T03:10:33.184 INFO:teuthology.orchestra.run.smithi146.stderr: count bytes template 2024-09-22T03:10:33.184 INFO:teuthology.orchestra.run.smithi146.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:33.184 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:33.184 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:10:33.184 INFO:teuthology.orchestra.run.smithi146.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:33.225 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-22T03:10:33.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:33.802 INFO:teuthology.orchestra.run.smithi146.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:33.803 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:33,802 seastar - Reactor backend: linux-aio 2024-09-22T03:10:33.821 INFO:teuthology.orchestra.run.smithi146.stderr:==27904==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:33.866 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:33,865 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:10:33.866 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:33,865 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-22T03:10:33.867 INFO:teuthology.orchestra.run.smithi146.stderr:DEBUG 2024-09-22 03:10:33,866 [shard 0:main] seastar - reactor::drain 2024-09-22T03:10:34.346 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:34.346 INFO:teuthology.orchestra.run.smithi146.stderr:Suppressions used: 2024-09-22T03:10:34.346 INFO:teuthology.orchestra.run.smithi146.stderr: count bytes template 2024-09-22T03:10:34.346 INFO:teuthology.orchestra.run.smithi146.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:34.346 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:34.346 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:10:34.399 INFO:teuthology.orchestra.run.smithi146.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:34.401 INFO:teuthology.orchestra.run.smithi146.stderr:INFO 2024-09-22 03:10:34,400 seastar - Reactor backend: linux-aio 2024-09-22T03:10:34.429 INFO:teuthology.orchestra.run.smithi146.stderr:==27902==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:39.530 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:39.530 INFO:teuthology.orchestra.run.smithi146.stderr:Suppressions used: 2024-09-22T03:10:39.530 INFO:teuthology.orchestra.run.smithi146.stderr: count bytes template 2024-09-22T03:10:39.530 INFO:teuthology.orchestra.run.smithi146.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:39.531 INFO:teuthology.orchestra.run.smithi146.stderr:----------------------------------------------------- 2024-09-22T03:10:39.531 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:10:39.531 INFO:teuthology.orchestra.run.smithi146.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:39.571 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-22T03:10:39.596 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-22T03:10:39.627 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:10:39.627 INFO:tasks.ceph:role: osd.2 2024-09-22T03:10:39.627 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-22T03:10:39.627 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T03:10:39.704 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:10:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:10:40.221 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-22T03:10:40.224 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-22T03:10:40.224 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-22T03:10:40.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-22T03:10:40.378 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-22T03:10:40.443 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:10:40.443 INFO:tasks.ceph:role: osd.3 2024-09-22T03:10:40.443 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-22T03:10:40.443 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T03:10:40.516 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:10:40.517 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:10:40.965 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-22T03:10:40.967 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-22T03:10:40.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-09-22T03:10:41.050 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-22T03:10:41.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:41.691 INFO:teuthology.orchestra.run.smithi148.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:41.693 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:41,691 seastar - Reactor backend: linux-aio 2024-09-22T03:10:41.710 INFO:teuthology.orchestra.run.smithi148.stderr:==27899==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:41.757 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:41,756 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:10:41.758 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:41,756 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-22T03:10:41.758 INFO:teuthology.orchestra.run.smithi148.stderr:DEBUG 2024-09-22 03:10:41,757 [shard 0:main] seastar - reactor::drain 2024-09-22T03:10:42.247 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:42.248 INFO:teuthology.orchestra.run.smithi148.stderr:Suppressions used: 2024-09-22T03:10:42.248 INFO:teuthology.orchestra.run.smithi148.stderr: count bytes template 2024-09-22T03:10:42.248 INFO:teuthology.orchestra.run.smithi148.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:42.248 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:42.248 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-22T03:10:42.304 INFO:teuthology.orchestra.run.smithi148.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:42.305 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:42,305 seastar - Reactor backend: linux-aio 2024-09-22T03:10:42.342 INFO:teuthology.orchestra.run.smithi148.stderr:==27896==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:47.420 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:47.420 INFO:teuthology.orchestra.run.smithi148.stderr:Suppressions used: 2024-09-22T03:10:47.421 INFO:teuthology.orchestra.run.smithi148.stderr: count bytes template 2024-09-22T03:10:47.421 INFO:teuthology.orchestra.run.smithi148.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:47.421 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:47.421 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-22T03:10:47.421 INFO:teuthology.orchestra.run.smithi148.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:47.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-22T03:10:47.484 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:48.032 INFO:teuthology.orchestra.run.smithi148.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:48.033 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:48,032 seastar - Reactor backend: linux-aio 2024-09-22T03:10:48.056 INFO:teuthology.orchestra.run.smithi148.stderr:==28085==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:48.101 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:48,100 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:10:48.101 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:48,100 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-22T03:10:48.102 INFO:teuthology.orchestra.run.smithi148.stderr:DEBUG 2024-09-22 03:10:48,101 [shard 0:main] seastar - reactor::drain 2024-09-22T03:10:48.585 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:48.586 INFO:teuthology.orchestra.run.smithi148.stderr:Suppressions used: 2024-09-22T03:10:48.586 INFO:teuthology.orchestra.run.smithi148.stderr: count bytes template 2024-09-22T03:10:48.586 INFO:teuthology.orchestra.run.smithi148.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:48.586 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:48.586 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-22T03:10:48.639 INFO:teuthology.orchestra.run.smithi148.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:10:48.641 INFO:teuthology.orchestra.run.smithi148.stderr:INFO 2024-09-22 03:10:48,640 seastar - Reactor backend: linux-aio 2024-09-22T03:10:48.676 INFO:teuthology.orchestra.run.smithi148.stderr:==28083==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:10:53.790 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:53.790 INFO:teuthology.orchestra.run.smithi148.stderr:Suppressions used: 2024-09-22T03:10:53.790 INFO:teuthology.orchestra.run.smithi148.stderr: count bytes template 2024-09-22T03:10:53.791 INFO:teuthology.orchestra.run.smithi148.stderr: 1 8 MallocExtension::Register 2024-09-22T03:10:53.791 INFO:teuthology.orchestra.run.smithi148.stderr:----------------------------------------------------- 2024-09-22T03:10:53.791 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-22T03:10:53.791 INFO:teuthology.orchestra.run.smithi148.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:10:53.830 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-22T03:10:53.854 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-22T03:10:53.854 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:53.854 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-22T03:10:53.884 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:53.885 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-22T03:10:53.954 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:53.955 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-22T03:10:53.980 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:53.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-22T03:10:54.045 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:54.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-22T03:10:54.110 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:54.110 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-22T03:10:54.129 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:54.129 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-22T03:10:54.166 INFO:tasks.ceph:Adding keys to all mons... 2024-09-22T03:10:54.166 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCkiu9mLRTqCRAAopB53F33zWt7t4+Eq1dOYQ== 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCqiu9mt1b1HBAA18OuD1VJZ7a3UoXRjq3XBw== 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQChiu9mRUc3DhAA+etMRrFMb50yJqb57Ik6pA== 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCyiu9mnbnpFxAAhPnYqQdBJ08jnK0Bqt/AvQ== 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2024-09-22T03:10:54.196 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQC4iu9mrPm/KxAABsNC5EdonXhT82yLShaJTQ== 2024-09-22T03:10:54.197 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-09-22T03:10:54.197 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQChiu9mAvk1ERAA0iKlrAjA0aNh56ZfbjkHhA== 2024-09-22T03:10:54.197 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2024-09-22T03:10:54.197 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQChiu9mhWbEFBAAS3iN+0vTiZ2wFcU8auV4eg== 2024-09-22T03:10:54.198 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:10:54.301 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:10:54.360 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T03:10:54.417 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:10:54.473 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:10:54.530 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T03:10:54.590 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T03:10:54.649 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-22T03:10:54.649 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-22T03:10:54.673 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T03:10:54.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:54.765+0000 7f5dc0e8cb40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:54.765+0000 7f5dc0e8cb40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:54.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-22T03:10:54.804 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:10:54.859 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-22T03:10:54.859 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-22T03:10:54.860 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-22T03:10:54.901 INFO:tasks.ceph.mon.a:Started 2024-09-22T03:10:54.902 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-22T03:10:54.902 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-22T03:10:54.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-22T03:10:54.904 INFO:tasks.ceph.mgr.x:Started 2024-09-22T03:10:54.904 INFO:tasks.ceph:Setting crush tunables to default 2024-09-22T03:10:54.905 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd crush tunables default 2024-09-22T03:10:54.979 INFO:tasks.ceph.mon.a.smithi146.stderr:2024-09-22T03:10:54.977+0000 7f2a2e039b40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:54.979 INFO:tasks.ceph.mon.a.smithi146.stderr:2024-09-22T03:10:54.977+0000 7f2a2e039b40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:54.979 INFO:tasks.ceph.mon.a.smithi146.stderr:2024-09-22T03:10:54.977+0000 7f2a2e039b40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:54.995 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:54.994+0000 7fed0e1c2f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:55.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:55.002+0000 7ff193d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:55.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:55.021+0000 7ff193d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:55.024 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.022+0000 7fed0e1c2f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:55.024 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.022+0000 7fed0e1c2f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:55.090 INFO:teuthology.orchestra.run.smithi146.stderr:adjusted tunables profile to default 2024-09-22T03:10:55.101 INFO:tasks.ceph:check_enable_crimson: True 2024-09-22T03:10:55.101 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-09-22T03:10:55.101 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-09-22T03:10:55.124 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.123+0000 7fed0e1c2f00 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-22T03:10:55.166 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.164+0000 7fed0e1c2f00 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-22T03:10:55.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:55.197+0000 7f6bf22d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:55.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:55.204+0000 7f6bf22d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:55.282 INFO:tasks.ceph.mon.a.smithi146.stderr:2024-09-22T03:10:55.281+0000 7f2a25ff1640 -1 WARNING: experimental feature 'crimson' is enabled 2024-09-22T03:10:55.283 INFO:tasks.ceph.mon.a.smithi146.stderr:Please be aware that this feature is experimental, untested, 2024-09-22T03:10:55.283 INFO:tasks.ceph.mon.a.smithi146.stderr:unsupported, and may result in data corruption, data loss, 2024-09-22T03:10:55.283 INFO:tasks.ceph.mon.a.smithi146.stderr:and/or irreparable damage to your cluster. Do not use 2024-09-22T03:10:55.283 INFO:tasks.ceph.mon.a.smithi146.stderr:feature with important data. 2024-09-22T03:10:55.283 INFO:tasks.ceph.mon.a.smithi146.stderr: 2024-09-22T03:10:55.302 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.300+0000 7fed0e1c2f00 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-22T03:10:55.352 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.351+0000 7fed0e1c2f00 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-22T03:10:55.506 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.504+0000 7fed0e1c2f00 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-22T03:10:55.615 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.614+0000 7fed0e1c2f00 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-22T03:10:55.939 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.937+0000 7fed0e1c2f00 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-22T03:10:55.986 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:55.984+0000 7fed0e1c2f00 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-22T03:10:56.104 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-22T03:10:56.104 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:56.104 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-22T03:10:56.131 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-22T03:10:56.131 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-22T03:10:56.135 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:56.133+0000 7fed0e1c2f00 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-22T03:10:56.198 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:56.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-22T03:10:56.223 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-22T03:10:56.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-22T03:10:56.288 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 2e9a0822-2cc0-4936-a416-074895e4a8c9 0 2024-09-22T03:10:56.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:56.425+0000 7f27200b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:56.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:56.433+0000 7f27200b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:56.511 INFO:teuthology.orchestra.run.smithi148.stdout:0 2024-09-22T03:10:56.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 838297aa-1306-4265-aebc-f5a153440ca8 1 2024-09-22T03:10:56.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:56.618+0000 7f562606c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:56.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:56.627+0000 7f562606c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:56.703 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-22T03:10:56.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 76e8173e-b2ac-46f1-9dbc-131bb527a587 2 2024-09-22T03:10:56.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:56.812+0000 7fd244397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:56.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:56.820+0000 7fd244397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:56.898 INFO:teuthology.orchestra.run.smithi148.stdout:2 2024-09-22T03:10:56.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new d32dd872-5134-4314-931d-4de8663e2c6d 3 2024-09-22T03:10:57.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:57.005+0000 7fe75c797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:57.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-22T03:10:57.014+0000 7fe75c797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:57.095 INFO:teuthology.orchestra.run.smithi148.stdout:3 2024-09-22T03:10:57.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-09-22T03:10:57.138 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:57.136+0000 7fed0e1c2f00 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-22T03:10:57.141 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:10:57.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:10:57.184 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:10:57.184 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-09-22T03:10:57.191 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:57.190+0000 7fed0e1c2f00 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-22T03:10:57.216 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:10:57.217 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:10:57.259 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:10:57.259 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-09-22T03:10:57.289 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:10:57.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:10:57.331 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:10:57.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-09-22T03:10:57.361 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:10:57.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:10:57.404 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:10:57.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:10:57.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:57.521+0000 7fc1fdb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:57.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:10:57.530+0000 7fc1fdb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:57.598 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:10:57.598 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":7,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:10:57.091220+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:10:57.608 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:57.606+0000 7fed0e1c2f00 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-22T03:10:57.612 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-22T03:10:57.612 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-22T03:10:57.712 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:57.710+0000 7fed0e1c2f00 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-22T03:10:57.836 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:57.834+0000 7fed0e1c2f00 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-22T03:10:57.887 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:57.886+0000 7fed0e1c2f00 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-22T03:10:58.009 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.007+0000 7fed0e1c2f00 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-22T03:10:58.014 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T03:10:58.064 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.063+0000 7fed0e1c2f00 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-22T03:10:58.115 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.113+0000 7fed0e1c2f00 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-22T03:10:58.116 INFO:teuthology.misc.health.smithi146.stderr:2024-09-22T03:10:58.111+0000 7f0136e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:58.119 INFO:teuthology.misc.health.smithi146.stderr:2024-09-22T03:10:58.118+0000 7f0136e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:10:58.187 INFO:teuthology.misc.health.smithi146.stdout: 2024-09-22T03:10:58.187 INFO:teuthology.misc.health.smithi146.stdout:{"epoch":7,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:10:57.091220+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:10:58.197 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-22T03:10:58.238 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.236+0000 7fed0e1c2f00 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-22T03:10:58.293 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.291+0000 7fed0e1c2f00 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-22T03:10:58.349 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.347+0000 7fed0e1c2f00 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-22T03:10:58.423 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.422+0000 7fed0e1c2f00 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-22T03:10:58.472 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.471+0000 7fed0e1c2f00 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-22T03:10:58.521 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:58.519+0000 7fed0e1c2f00 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-22T03:10:59.215 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:10:59.213+0000 7fed0e1c2f00 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-22T03:11:00.227 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-22T03:11:00.226+0000 7fecac90b640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-22T03:11:04.601 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T03:11:04.709 INFO:teuthology.misc.health.smithi146.stderr:2024-09-22T03:11:04.702+0000 7fb7a7397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:04.712 INFO:teuthology.misc.health.smithi146.stderr:2024-09-22T03:11:04.709+0000 7fb7a7397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:04.889 INFO:teuthology.misc.health.smithi146.stdout: 2024-09-22T03:11:04.890 INFO:teuthology.misc.health.smithi146.stdout:{"epoch":10,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:11:03.231542+0000","last_up_change":"2024-09-22T03:11:03.231542+0000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":487370652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":487370652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":487370652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":487370652}]},"public_addr":"172.21.15.146:6802/487370652","cluster_addr":"172.21.15.146:6801/487370652","heartbeat_back_addr":"172.21.15.146:6806/487370652","heartbeat_front_addr":"172.21.15.146:6807/487370652","state":["exists","up"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1187154252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1187154252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":1187154252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1187154252}]},"public_addr":"172.21.15.146:6805/1187154252","cluster_addr":"172.21.15.146:6804/1187154252","heartbeat_back_addr":"172.21.15.146:6803/1187154252","heartbeat_front_addr":"172.21.15.146:6808/1187154252","state":["exists","up"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6805","nonce":756085572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":756085572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":756085572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":756085572}]},"public_addr":"172.21.15.148:6805/756085572","cluster_addr":"172.21.15.148:6804/756085572","heartbeat_back_addr":"172.21.15.148:6812/756085572","heartbeat_front_addr":"172.21.15.148:6813/756085572","state":["exists","up"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1572191826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":1572191826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6811","nonce":1572191826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1572191826}]},"public_addr":"172.21.15.148:6808/1572191826","cluster_addr":"172.21.15.148:6809/1572191826","heartbeat_back_addr":"172.21.15.148:6811/1572191826","heartbeat_front_addr":"172.21.15.148:6810/1572191826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:11:04.901 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-22T03:11:04.901 INFO:tasks.ceph:Creating RBD pool 2024-09-22T03:11:04.902 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-22T03:11:05.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:04.998+0000 7f403d0c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:05.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:05.007+0000 7f403d0c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:05.257 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'rbd' created 2024-09-22T03:11:05.273 DEBUG:teuthology.orchestra.run.smithi146:> rbd --cluster ceph pool init rbd 2024-09-22T03:11:05.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:05.333+0000 7f13aa030d00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:05.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:05.333+0000 7f13aa030d00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:05.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:05.337+0000 7f13aa030d00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:08.293 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-22T03:11:08.293 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-22T03:11:08.294 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-22T03:11:08.294 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-22T03:11:08.294 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-22T03:11:08.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:08.396+0000 7ffa15957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:08.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:08.402+0000 7ffa15957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:08.629 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:08.641 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1148763145},{"type":"v1","addr":"172.21.15.148:6801","nonce":1148763145}]},"active_addr":"172.21.15.148:6801/1148763145","active_change":"2024-09-22T03:10:59.216017+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":2753208183}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":306210735}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3035916553}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":246959598}]}]} 2024-09-22T03:11:08.643 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-22T03:11:08.643 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T03:11:08.643 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:11:08.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:08.740+0000 7f806dd0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:08.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:08.747+0000 7f806dd0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:08.926 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:08.926 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":14,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:11:08.260475+0000","last_up_change":"2024-09-22T03:11:03.231542+0000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-22T03:11:05.202857+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-22T03:11:05.252894+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":487370652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":487370652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":487370652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":487370652}]},"public_addr":"172.21.15.146:6802/487370652","cluster_addr":"172.21.15.146:6801/487370652","heartbeat_back_addr":"172.21.15.146:6806/487370652","heartbeat_front_addr":"172.21.15.146:6807/487370652","state":["exists","up"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1187154252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1187154252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":1187154252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1187154252}]},"public_addr":"172.21.15.146:6805/1187154252","cluster_addr":"172.21.15.146:6804/1187154252","heartbeat_back_addr":"172.21.15.146:6803/1187154252","heartbeat_front_addr":"172.21.15.146:6808/1187154252","state":["exists","up"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6805","nonce":756085572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":756085572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":756085572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":756085572}]},"public_addr":"172.21.15.148:6805/756085572","cluster_addr":"172.21.15.148:6804/756085572","heartbeat_back_addr":"172.21.15.148:6812/756085572","heartbeat_front_addr":"172.21.15.148:6813/756085572","state":["exists","up"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1572191826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":1572191826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6811","nonce":1572191826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1572191826}]},"public_addr":"172.21.15.148:6808/1572191826","cluster_addr":"172.21.15.148:6809/1572191826","heartbeat_back_addr":"172.21.15.148:6811/1572191826","heartbeat_front_addr":"172.21.15.148:6810/1572191826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:11:08.938 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T03:11:08.939 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:11:09.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.037+0000 7f4276aba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.044+0000 7f4276aba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.224 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:09.224 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":14,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:11:08.260475+0000","last_up_change":"2024-09-22T03:11:03.231542+0000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-22T03:11:05.202857+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-22T03:11:05.252894+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":487370652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":487370652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":487370652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":487370652}]},"public_addr":"172.21.15.146:6802/487370652","cluster_addr":"172.21.15.146:6801/487370652","heartbeat_back_addr":"172.21.15.146:6806/487370652","heartbeat_front_addr":"172.21.15.146:6807/487370652","state":["exists","up"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1187154252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1187154252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":1187154252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1187154252}]},"public_addr":"172.21.15.146:6805/1187154252","cluster_addr":"172.21.15.146:6804/1187154252","heartbeat_back_addr":"172.21.15.146:6803/1187154252","heartbeat_front_addr":"172.21.15.146:6808/1187154252","state":["exists","up"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6805","nonce":756085572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":756085572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":756085572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":756085572}]},"public_addr":"172.21.15.148:6805/756085572","cluster_addr":"172.21.15.148:6804/756085572","heartbeat_back_addr":"172.21.15.148:6812/756085572","heartbeat_front_addr":"172.21.15.148:6813/756085572","state":["exists","up"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1572191826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":1572191826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6811","nonce":1572191826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1572191826}]},"public_addr":"172.21.15.148:6808/1572191826","cluster_addr":"172.21.15.148:6809/1572191826","heartbeat_back_addr":"172.21.15.148:6811/1572191826","heartbeat_front_addr":"172.21.15.148:6810/1572191826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:11:09.238 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T03:11:09.238 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T03:11:09.238 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T03:11:09.239 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T03:11:09.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.348+0000 7f66dcc4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.349+0000 7f4a97e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.348+0000 7f83812fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.348+0000 7f55d8557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.357+0000 7f4a97e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.357+0000 7f83812fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.358+0000 7f66dcc4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.358+0000 7f55d8557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.390 INFO:teuthology.orchestra.run.smithi146.stdout:42949672966 2024-09-22T03:11:09.390 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:11:09.391 INFO:teuthology.orchestra.run.smithi146.stdout:38654705671 2024-09-22T03:11:09.392 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:11:09.393 INFO:teuthology.orchestra.run.smithi146.stdout:42949672966 2024-09-22T03:11:09.394 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:11:09.395 INFO:teuthology.orchestra.run.smithi146.stdout:42949672966 2024-09-22T03:11:09.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:11:09.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.495+0000 7fa92c157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.505+0000 7fa92c157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.543+0000 7f04d7797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.543+0000 7fb55e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.543+0000 7f9644d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.552+0000 7f04d7797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.552+0000 7fb55e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.552+0000 7f9644d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.706 INFO:teuthology.orchestra.run.smithi146.stdout:42949672966 2024-09-22T03:11:09.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.3 2024-09-22T03:11:09.720 DEBUG:teuthology.parallel:result is None 2024-09-22T03:11:09.743 INFO:teuthology.orchestra.run.smithi146.stdout:38654705671 2024-09-22T03:11:09.753 INFO:teuthology.orchestra.run.smithi146.stdout:42949672966 2024-09-22T03:11:09.754 INFO:teuthology.orchestra.run.smithi146.stdout:42949672966 2024-09-22T03:11:09.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.0 2024-09-22T03:11:09.756 DEBUG:teuthology.parallel:result is None 2024-09-22T03:11:09.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.2 2024-09-22T03:11:09.767 DEBUG:teuthology.parallel:result is None 2024-09-22T03:11:09.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.1 2024-09-22T03:11:09.768 DEBUG:teuthology.parallel:result is None 2024-09-22T03:11:09.768 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T03:11:09.768 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:09.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.906+0000 7f143b4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:09.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:09.913+0000 7f143b4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:10.092 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:10.092 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:10.105 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-22T03:11:09.379152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108020,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379340,"statfs":{"total":386547056640,"available":386436444160,"internally_reserved":0,"allocated":671744,"data_stored":1894113,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.121012"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278861+0000","last_change":"2024-09-22T03:11:08.280433+0000","last_active":"2024-09-22T03:11:08.278861+0000","last_peered":"2024-09-22T03:11:08.278861+0000","last_clean":"2024-09-22T03:11:08.278861+0000","last_became_active":"2024-09-22T03:11:06.302054+0000","last_became_peered":"2024-09-22T03:11:06.302054+0000","last_unstale":"2024-09-22T03:11:08.278861+0000","last_undegraded":"2024-09-22T03:11:08.278861+0000","last_fullsized":"2024-09-22T03:11:08.278861+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279241+0000","last_change":"2024-09-22T03:11:08.280642+0000","last_active":"2024-09-22T03:11:08.279241+0000","last_peered":"2024-09-22T03:11:08.279241+0000","last_clean":"2024-09-22T03:11:08.279241+0000","last_became_active":"2024-09-22T03:11:06.332949+0000","last_became_peered":"2024-09-22T03:11:06.332949+0000","last_unstale":"2024-09-22T03:11:08.279241+0000","last_undegraded":"2024-09-22T03:11:08.279241+0000","last_fullsized":"2024-09-22T03:11:08.279241+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279079+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:08.279079+0000","last_peered":"2024-09-22T03:11:08.279079+0000","last_clean":"2024-09-22T03:11:08.279079+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:08.279079+0000","last_undegraded":"2024-09-22T03:11:08.279079+0000","last_fullsized":"2024-09-22T03:11:08.279079+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276778+0000","last_change":"2024-09-22T03:11:08.278389+0000","last_active":"2024-09-22T03:11:08.276778+0000","last_peered":"2024-09-22T03:11:08.276778+0000","last_clean":"2024-09-22T03:11:08.276778+0000","last_became_active":"2024-09-22T03:11:06.309361+0000","last_became_peered":"2024-09-22T03:11:06.309361+0000","last_unstale":"2024-09-22T03:11:08.276778+0000","last_undegraded":"2024-09-22T03:11:08.276778+0000","last_fullsized":"2024-09-22T03:11:08.276778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":142,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279535+0000","last_change":"2024-09-22T03:11:06.330965+0000","last_active":"2024-09-22T03:11:08.279535+0000","last_peered":"2024-09-22T03:11:08.279535+0000","last_clean":"2024-09-22T03:11:08.279535+0000","last_became_active":"2024-09-22T03:11:06.324553+0000","last_became_peered":"2024-09-22T03:11:06.324553+0000","last_unstale":"2024-09-22T03:11:08.279535+0000","last_undegraded":"2024-09-22T03:11:08.279535+0000","last_fullsized":"2024-09-22T03:11:08.279535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":17,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278894+0000","last_change":"2024-09-22T03:11:08.280339+0000","last_active":"2024-09-22T03:11:08.278894+0000","last_peered":"2024-09-22T03:11:08.278894+0000","last_clean":"2024-09-22T03:11:08.278894+0000","last_became_active":"2024-09-22T03:11:06.303791+0000","last_became_peered":"2024-09-22T03:11:06.303791+0000","last_unstale":"2024-09-22T03:11:08.278894+0000","last_undegraded":"2024-09-22T03:11:08.278894+0000","last_fullsized":"2024-09-22T03:11:08.278894+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"13'1","reported_seq":18,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278465+0000","last_change":"2024-09-22T03:11:08.278465+0000","last_active":"2024-09-22T03:11:08.278465+0000","last_peered":"2024-09-22T03:11:08.278465+0000","last_clean":"2024-09-22T03:11:08.278465+0000","last_became_active":"2024-09-22T03:11:06.312069+0000","last_became_peered":"2024-09-22T03:11:06.312069+0000","last_unstale":"2024-09-22T03:11:08.278465+0000","last_undegraded":"2024-09-22T03:11:08.278465+0000","last_fullsized":"2024-09-22T03:11:08.278465+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276680+0000","last_change":"2024-09-22T03:11:08.279144+0000","last_active":"2024-09-22T03:11:08.276680+0000","last_peered":"2024-09-22T03:11:08.276680+0000","last_clean":"2024-09-22T03:11:08.276680+0000","last_became_active":"2024-09-22T03:11:06.307515+0000","last_became_peered":"2024-09-22T03:11:06.307515+0000","last_unstale":"2024-09-22T03:11:08.276680+0000","last_undegraded":"2024-09-22T03:11:08.276680+0000","last_fullsized":"2024-09-22T03:11:08.276680+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276445+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:08.276445+0000","last_peered":"2024-09-22T03:11:08.276445+0000","last_clean":"2024-09-22T03:11:08.276445+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:08.276445+0000","last_undegraded":"2024-09-22T03:11:08.276445+0000","last_fullsized":"2024-09-22T03:11:08.276445+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":122880,"data_stored":30757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":180224,"data_stored":621106,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:10.106 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:10.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:10.206+0000 7f475a6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:10.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:10.213+0000 7f475a6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:10.393 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:10.394 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:10.406 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-22T03:11:09.379152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108020,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379340,"statfs":{"total":386547056640,"available":386436444160,"internally_reserved":0,"allocated":671744,"data_stored":1894113,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.121012"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278861+0000","last_change":"2024-09-22T03:11:08.280433+0000","last_active":"2024-09-22T03:11:08.278861+0000","last_peered":"2024-09-22T03:11:08.278861+0000","last_clean":"2024-09-22T03:11:08.278861+0000","last_became_active":"2024-09-22T03:11:06.302054+0000","last_became_peered":"2024-09-22T03:11:06.302054+0000","last_unstale":"2024-09-22T03:11:08.278861+0000","last_undegraded":"2024-09-22T03:11:08.278861+0000","last_fullsized":"2024-09-22T03:11:08.278861+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279241+0000","last_change":"2024-09-22T03:11:08.280642+0000","last_active":"2024-09-22T03:11:08.279241+0000","last_peered":"2024-09-22T03:11:08.279241+0000","last_clean":"2024-09-22T03:11:08.279241+0000","last_became_active":"2024-09-22T03:11:06.332949+0000","last_became_peered":"2024-09-22T03:11:06.332949+0000","last_unstale":"2024-09-22T03:11:08.279241+0000","last_undegraded":"2024-09-22T03:11:08.279241+0000","last_fullsized":"2024-09-22T03:11:08.279241+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279079+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:08.279079+0000","last_peered":"2024-09-22T03:11:08.279079+0000","last_clean":"2024-09-22T03:11:08.279079+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:08.279079+0000","last_undegraded":"2024-09-22T03:11:08.279079+0000","last_fullsized":"2024-09-22T03:11:08.279079+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276778+0000","last_change":"2024-09-22T03:11:08.278389+0000","last_active":"2024-09-22T03:11:08.276778+0000","last_peered":"2024-09-22T03:11:08.276778+0000","last_clean":"2024-09-22T03:11:08.276778+0000","last_became_active":"2024-09-22T03:11:06.309361+0000","last_became_peered":"2024-09-22T03:11:06.309361+0000","last_unstale":"2024-09-22T03:11:08.276778+0000","last_undegraded":"2024-09-22T03:11:08.276778+0000","last_fullsized":"2024-09-22T03:11:08.276778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":142,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279535+0000","last_change":"2024-09-22T03:11:06.330965+0000","last_active":"2024-09-22T03:11:08.279535+0000","last_peered":"2024-09-22T03:11:08.279535+0000","last_clean":"2024-09-22T03:11:08.279535+0000","last_became_active":"2024-09-22T03:11:06.324553+0000","last_became_peered":"2024-09-22T03:11:06.324553+0000","last_unstale":"2024-09-22T03:11:08.279535+0000","last_undegraded":"2024-09-22T03:11:08.279535+0000","last_fullsized":"2024-09-22T03:11:08.279535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":17,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278894+0000","last_change":"2024-09-22T03:11:08.280339+0000","last_active":"2024-09-22T03:11:08.278894+0000","last_peered":"2024-09-22T03:11:08.278894+0000","last_clean":"2024-09-22T03:11:08.278894+0000","last_became_active":"2024-09-22T03:11:06.303791+0000","last_became_peered":"2024-09-22T03:11:06.303791+0000","last_unstale":"2024-09-22T03:11:08.278894+0000","last_undegraded":"2024-09-22T03:11:08.278894+0000","last_fullsized":"2024-09-22T03:11:08.278894+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"13'1","reported_seq":18,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278465+0000","last_change":"2024-09-22T03:11:08.278465+0000","last_active":"2024-09-22T03:11:08.278465+0000","last_peered":"2024-09-22T03:11:08.278465+0000","last_clean":"2024-09-22T03:11:08.278465+0000","last_became_active":"2024-09-22T03:11:06.312069+0000","last_became_peered":"2024-09-22T03:11:06.312069+0000","last_unstale":"2024-09-22T03:11:08.278465+0000","last_undegraded":"2024-09-22T03:11:08.278465+0000","last_fullsized":"2024-09-22T03:11:08.278465+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276680+0000","last_change":"2024-09-22T03:11:08.279144+0000","last_active":"2024-09-22T03:11:08.276680+0000","last_peered":"2024-09-22T03:11:08.276680+0000","last_clean":"2024-09-22T03:11:08.276680+0000","last_became_active":"2024-09-22T03:11:06.307515+0000","last_became_peered":"2024-09-22T03:11:06.307515+0000","last_unstale":"2024-09-22T03:11:08.276680+0000","last_undegraded":"2024-09-22T03:11:08.276680+0000","last_fullsized":"2024-09-22T03:11:08.276680+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276445+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:08.276445+0000","last_peered":"2024-09-22T03:11:08.276445+0000","last_clean":"2024-09-22T03:11:08.276445+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:08.276445+0000","last_undegraded":"2024-09-22T03:11:08.276445+0000","last_fullsized":"2024-09-22T03:11:08.276445+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":122880,"data_stored":30757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":180224,"data_stored":621106,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:10.407 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T03:11:10.407 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-22T03:11:10.407 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-22T03:11:10.407 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-22T03:11:10.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:10.505+0000 7fbef6f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:10.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:10.510+0000 7fbef6f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:10.722 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:10.722 INFO:teuthology.orchestra.run.smithi146.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-22T03:11:10.735 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-22T03:11:10.735 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-22T03:11:10.750 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-22T03:11:10.750 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'squid', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T03:11:10.750 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-22T03:11:10.751 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-22T03:11:10.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:10.848+0000 7f1a054ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:10.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:10.858+0000 7f1a054ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:11.050 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-22T03:11:11.051 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T03:11:11.051 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:11.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:11.150+0000 7f9c07ece640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:11.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:11.157+0000 7f9c07ece640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:11.339 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:11.351 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-22T03:11:11.223232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108020,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379340,"statfs":{"total":386547056640,"available":386436444160,"internally_reserved":0,"allocated":671744,"data_stored":1894113,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.965093"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278861+0000","last_change":"2024-09-22T03:11:08.280433+0000","last_active":"2024-09-22T03:11:08.278861+0000","last_peered":"2024-09-22T03:11:08.278861+0000","last_clean":"2024-09-22T03:11:08.278861+0000","last_became_active":"2024-09-22T03:11:06.302054+0000","last_became_peered":"2024-09-22T03:11:06.302054+0000","last_unstale":"2024-09-22T03:11:08.278861+0000","last_undegraded":"2024-09-22T03:11:08.278861+0000","last_fullsized":"2024-09-22T03:11:08.278861+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279241+0000","last_change":"2024-09-22T03:11:08.280642+0000","last_active":"2024-09-22T03:11:08.279241+0000","last_peered":"2024-09-22T03:11:08.279241+0000","last_clean":"2024-09-22T03:11:08.279241+0000","last_became_active":"2024-09-22T03:11:06.332949+0000","last_became_peered":"2024-09-22T03:11:06.332949+0000","last_unstale":"2024-09-22T03:11:08.279241+0000","last_undegraded":"2024-09-22T03:11:08.279241+0000","last_fullsized":"2024-09-22T03:11:08.279241+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279079+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:08.279079+0000","last_peered":"2024-09-22T03:11:08.279079+0000","last_clean":"2024-09-22T03:11:08.279079+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:08.279079+0000","last_undegraded":"2024-09-22T03:11:08.279079+0000","last_fullsized":"2024-09-22T03:11:08.279079+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276778+0000","last_change":"2024-09-22T03:11:08.278389+0000","last_active":"2024-09-22T03:11:08.276778+0000","last_peered":"2024-09-22T03:11:08.276778+0000","last_clean":"2024-09-22T03:11:08.276778+0000","last_became_active":"2024-09-22T03:11:06.309361+0000","last_became_peered":"2024-09-22T03:11:06.309361+0000","last_unstale":"2024-09-22T03:11:08.276778+0000","last_undegraded":"2024-09-22T03:11:08.276778+0000","last_fullsized":"2024-09-22T03:11:08.276778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":142,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279535+0000","last_change":"2024-09-22T03:11:06.330965+0000","last_active":"2024-09-22T03:11:08.279535+0000","last_peered":"2024-09-22T03:11:08.279535+0000","last_clean":"2024-09-22T03:11:08.279535+0000","last_became_active":"2024-09-22T03:11:06.324553+0000","last_became_peered":"2024-09-22T03:11:06.324553+0000","last_unstale":"2024-09-22T03:11:08.279535+0000","last_undegraded":"2024-09-22T03:11:08.279535+0000","last_fullsized":"2024-09-22T03:11:08.279535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":17,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278894+0000","last_change":"2024-09-22T03:11:08.280339+0000","last_active":"2024-09-22T03:11:08.278894+0000","last_peered":"2024-09-22T03:11:08.278894+0000","last_clean":"2024-09-22T03:11:08.278894+0000","last_became_active":"2024-09-22T03:11:06.303791+0000","last_became_peered":"2024-09-22T03:11:06.303791+0000","last_unstale":"2024-09-22T03:11:08.278894+0000","last_undegraded":"2024-09-22T03:11:08.278894+0000","last_fullsized":"2024-09-22T03:11:08.278894+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"13'1","reported_seq":18,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278465+0000","last_change":"2024-09-22T03:11:08.278465+0000","last_active":"2024-09-22T03:11:08.278465+0000","last_peered":"2024-09-22T03:11:08.278465+0000","last_clean":"2024-09-22T03:11:08.278465+0000","last_became_active":"2024-09-22T03:11:06.312069+0000","last_became_peered":"2024-09-22T03:11:06.312069+0000","last_unstale":"2024-09-22T03:11:08.278465+0000","last_undegraded":"2024-09-22T03:11:08.278465+0000","last_fullsized":"2024-09-22T03:11:08.278465+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276680+0000","last_change":"2024-09-22T03:11:08.279144+0000","last_active":"2024-09-22T03:11:08.276680+0000","last_peered":"2024-09-22T03:11:08.276680+0000","last_clean":"2024-09-22T03:11:08.276680+0000","last_became_active":"2024-09-22T03:11:06.307515+0000","last_became_peered":"2024-09-22T03:11:06.307515+0000","last_unstale":"2024-09-22T03:11:08.276680+0000","last_undegraded":"2024-09-22T03:11:08.276680+0000","last_fullsized":"2024-09-22T03:11:08.276680+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276445+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:08.276445+0000","last_peered":"2024-09-22T03:11:08.276445+0000","last_clean":"2024-09-22T03:11:08.276445+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:08.276445+0000","last_undegraded":"2024-09-22T03:11:08.276445+0000","last_fullsized":"2024-09-22T03:11:08.276445+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":122880,"data_stored":30757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":180224,"data_stored":621106,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:11.352 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:11.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:11.450+0000 7f3a55597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:11.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:11.457+0000 7f3a55597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:11.636 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:11.636 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:11.648 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-22T03:11:11.223232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108020,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379340,"statfs":{"total":386547056640,"available":386436444160,"internally_reserved":0,"allocated":671744,"data_stored":1894113,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.965093"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278861+0000","last_change":"2024-09-22T03:11:08.280433+0000","last_active":"2024-09-22T03:11:08.278861+0000","last_peered":"2024-09-22T03:11:08.278861+0000","last_clean":"2024-09-22T03:11:08.278861+0000","last_became_active":"2024-09-22T03:11:06.302054+0000","last_became_peered":"2024-09-22T03:11:06.302054+0000","last_unstale":"2024-09-22T03:11:08.278861+0000","last_undegraded":"2024-09-22T03:11:08.278861+0000","last_fullsized":"2024-09-22T03:11:08.278861+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279241+0000","last_change":"2024-09-22T03:11:08.280642+0000","last_active":"2024-09-22T03:11:08.279241+0000","last_peered":"2024-09-22T03:11:08.279241+0000","last_clean":"2024-09-22T03:11:08.279241+0000","last_became_active":"2024-09-22T03:11:06.332949+0000","last_became_peered":"2024-09-22T03:11:06.332949+0000","last_unstale":"2024-09-22T03:11:08.279241+0000","last_undegraded":"2024-09-22T03:11:08.279241+0000","last_fullsized":"2024-09-22T03:11:08.279241+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279079+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:08.279079+0000","last_peered":"2024-09-22T03:11:08.279079+0000","last_clean":"2024-09-22T03:11:08.279079+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:08.279079+0000","last_undegraded":"2024-09-22T03:11:08.279079+0000","last_fullsized":"2024-09-22T03:11:08.279079+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276778+0000","last_change":"2024-09-22T03:11:08.278389+0000","last_active":"2024-09-22T03:11:08.276778+0000","last_peered":"2024-09-22T03:11:08.276778+0000","last_clean":"2024-09-22T03:11:08.276778+0000","last_became_active":"2024-09-22T03:11:06.309361+0000","last_became_peered":"2024-09-22T03:11:06.309361+0000","last_unstale":"2024-09-22T03:11:08.276778+0000","last_undegraded":"2024-09-22T03:11:08.276778+0000","last_fullsized":"2024-09-22T03:11:08.276778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":142,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.279535+0000","last_change":"2024-09-22T03:11:06.330965+0000","last_active":"2024-09-22T03:11:08.279535+0000","last_peered":"2024-09-22T03:11:08.279535+0000","last_clean":"2024-09-22T03:11:08.279535+0000","last_became_active":"2024-09-22T03:11:06.324553+0000","last_became_peered":"2024-09-22T03:11:06.324553+0000","last_unstale":"2024-09-22T03:11:08.279535+0000","last_undegraded":"2024-09-22T03:11:08.279535+0000","last_fullsized":"2024-09-22T03:11:08.279535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":17,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278894+0000","last_change":"2024-09-22T03:11:08.280339+0000","last_active":"2024-09-22T03:11:08.278894+0000","last_peered":"2024-09-22T03:11:08.278894+0000","last_clean":"2024-09-22T03:11:08.278894+0000","last_became_active":"2024-09-22T03:11:06.303791+0000","last_became_peered":"2024-09-22T03:11:06.303791+0000","last_unstale":"2024-09-22T03:11:08.278894+0000","last_undegraded":"2024-09-22T03:11:08.278894+0000","last_fullsized":"2024-09-22T03:11:08.278894+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"13'1","reported_seq":18,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.278465+0000","last_change":"2024-09-22T03:11:08.278465+0000","last_active":"2024-09-22T03:11:08.278465+0000","last_peered":"2024-09-22T03:11:08.278465+0000","last_clean":"2024-09-22T03:11:08.278465+0000","last_became_active":"2024-09-22T03:11:06.312069+0000","last_became_peered":"2024-09-22T03:11:06.312069+0000","last_unstale":"2024-09-22T03:11:08.278465+0000","last_undegraded":"2024-09-22T03:11:08.278465+0000","last_fullsized":"2024-09-22T03:11:08.278465+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276680+0000","last_change":"2024-09-22T03:11:08.279144+0000","last_active":"2024-09-22T03:11:08.276680+0000","last_peered":"2024-09-22T03:11:08.276680+0000","last_clean":"2024-09-22T03:11:08.276680+0000","last_became_active":"2024-09-22T03:11:06.307515+0000","last_became_peered":"2024-09-22T03:11:06.307515+0000","last_unstale":"2024-09-22T03:11:08.276680+0000","last_undegraded":"2024-09-22T03:11:08.276680+0000","last_fullsized":"2024-09-22T03:11:08.276680+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-22T03:11:08.276445+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:08.276445+0000","last_peered":"2024-09-22T03:11:08.276445+0000","last_clean":"2024-09-22T03:11:08.276445+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:08.276445+0000","last_undegraded":"2024-09-22T03:11:08.276445+0000","last_fullsized":"2024-09-22T03:11:08.276445+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":122880,"data_stored":30757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":180224,"data_stored":621106,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:11.649 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T03:11:11.649 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-22T03:11:11.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:11.746+0000 7f3cf8397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:11.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:11.753+0000 7f3cf8397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:epoch 14 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:fsid 39bf19db-1c34-4685-8c41-ee47ed066388 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:created 2024-09-22T03:10:55.010188+0000 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:modified 2024-09-22T03:11:08.260475+0000 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:crush_version 4 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:full_ratio 0.95 2024-09-22T03:11:11.932 INFO:teuthology.orchestra.run.smithi146.stdout:backfillfull_ratio 0.9 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:nearfull_ratio 0.85 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:require_min_compat_client luminous 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:min_compat_client jewel 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:require_osd_release squid 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:stretch_mode_enabled false 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:allow_crimson=true 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,nopgchange,selfmanaged_snaps,crimson stripe_width 0 application rbd read_balance_score 1.50 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout: removed_snaps_queue [2~1] 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:pool 2 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 13 flags hashpspool,nopgchange,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:max_osd 4 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.146:6802/487370652 v2:172.21.15.146:6801/487370652 exists,up 2e9a0822-2cc0-4936-a416-074895e4a8c9 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:osd.1 up in weight 1 up_from 10 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.146:6805/1187154252 v2:172.21.15.146:6804/1187154252 exists,up 838297aa-1306-4265-aebc-f5a153440ca8 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:osd.2 up in weight 1 up_from 10 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.148:6805/756085572 v2:172.21.15.148:6804/756085572 exists,up 76e8173e-b2ac-46f1-9dbc-131bb527a587 2024-09-22T03:11:11.933 INFO:teuthology.orchestra.run.smithi146.stdout:osd.3 up in weight 1 up_from 10 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.148:6808/1572191826 v2:172.21.15.148:6809/1572191826 exists,up d32dd872-5134-4314-931d-4de8663e2c6d 2024-09-22T03:11:11.945 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.146:6802/487370652 v2:172.21.15.146:6801/487370652 exists,up 2e9a0822-2cc0-4936-a416-074895e4a8c9', 'osd.1 up in weight 1 up_from 10 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.146:6805/1187154252 v2:172.21.15.146:6804/1187154252 exists,up 838297aa-1306-4265-aebc-f5a153440ca8', 'osd.2 up in weight 1 up_from 10 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.148:6805/756085572 v2:172.21.15.148:6804/756085572 exists,up 76e8173e-b2ac-46f1-9dbc-131bb527a587', 'osd.3 up in weight 1 up_from 10 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.148:6808/1572191826 v2:172.21.15.148:6809/1572191826 exists,up d32dd872-5134-4314-931d-4de8663e2c6d'] 2024-09-22T03:11:11.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-22T03:11:12.049 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "mon.a", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster": "ceph", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "admin_socket_mode": "", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_client_required": "cephx, none", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_cluster_required": "cephx", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_debug": "true", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_service_required": "cephx", 2024-09-22T03:11:12.050 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_supported": "cephx", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_aio": "true", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_aio_poll_ms": "250", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_aio_reap_max": "16", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_async_discard": "false", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_block_size": "4096", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_debug_aio": "false", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_debug_inflight_ios": "false", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_enable_discard": "false", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_flock_retry": "3", 2024-09-22T03:11:12.051 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_inject_crash": "0", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_ioring": "false", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_ioring_hipri": "false", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bdev_type": "", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_alloc_size": "1048576", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_allocator": "hybrid", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_buffered_io": "true", 2024-09-22T03:11:12.052 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_check_for_zeros": "false", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_compact_log_sync": "false", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_max_log_runway": "4194304", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_max_prefetch": "1048576", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_min_flush_size": "524288", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_min_log_runway": "1048576", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_replay_recovery": "false", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-22T03:11:12.053 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_sync_write": "false", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_allocation_from_file": "true", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_allocator": "hybrid", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bdev_label_multi": "true", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-22T03:11:12.054 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_create": "true", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_db_create": "false", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_db_path": "", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_db_size": "0", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_path": "", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_preallocate_file": "false", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_size": "107374182400", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_wal_create": "false", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_wal_path": "", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_block_wal_size": "100663296", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bluefs": "true", 2024-09-22T03:11:12.055 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_autotune": "true", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_size": "0", 2024-09-22T03:11:12.056 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_type": "2q", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_clone_cow": "true", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-22T03:11:12.057 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_mode": "none", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_csum_type": "crc32c", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_freelist": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-22T03:11:12.058 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_small_allocations": "0", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_default_buffered_read": "true", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_default_buffered_write": "false", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-22T03:11:12.059 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fail_eio": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_on_mount": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_on_umount": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-22T03:11:12.060 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_ignore_data_csum": "false", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_log_op_age": "5.000000", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_max_alloc_size": "0", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_max_blob_size": "0", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-22T03:11:12.061 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_max_deferred_txc": "32", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_min_alloc_size": "0", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_nid_prealloc": "1024", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_retry_disk_reads": "3", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_rocksdb_cf": "true", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-22T03:11:12.062 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_spdk_mem": "512", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-22T03:11:12.063 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_tracing": "false", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_zero_block_detection": "false", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-22T03:11:12.064 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephx_cluster_require_signatures": "false", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephx_cluster_require_version": "2", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephx_require_signatures": "false", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephx_require_version": "2", 2024-09-22T03:11:12.065 INFO:teuthology.orchestra.run.smithi146.stdout: "cephx_service_require_signatures": "false", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "cephx_service_require_version": "2", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "cephx_sign_messages": "true", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "chdir": "", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_acl_type": "", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_asio_thread_count": "2", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_cache_mid": "0.750000", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_cache_size": "16384", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_caps_release_delay": "5", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_check_pool_perm": "true", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_debug_force_sync_read": "false", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_debug_getattr_caps": "false", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_debug_inject_features": "", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_debug_inject_tick_delay": "0", 2024-09-22T03:11:12.066 INFO:teuthology.orchestra.run.smithi146.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_die_on_failed_remount": "false", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_dirsize_rbytes": "true", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_force_lazyio": "false", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_fs": "", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_inject_release_failure": "false", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_max_inline_size": "4096", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_mds_namespace": "", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_metadata": "", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_mount_gid": "-1", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_mount_timeout": "300", 2024-09-22T03:11:12.067 INFO:teuthology.orchestra.run.smithi146.stdout: "client_mount_uid": "-1", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_mountpoint": "/", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_notify_timeout": "10", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_oc": "true", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_oc_max_dirty": "104857600", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_oc_max_objects": "1000", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_oc_size": "209715200", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_oc_target_dirty": "8388608", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_permissions": "true", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_quota": "true", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_quota_df": "true", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_readahead_max_bytes": "0", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_readahead_max_periods": "4", 2024-09-22T03:11:12.068 INFO:teuthology.orchestra.run.smithi146.stdout: "client_readahead_min": "131072", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_reconnect_stale": "false", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_shutdown_timeout": "30", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_snapdir": ".snap", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_tick_interval": "1", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_trace": "", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_try_dentry_invalidate": "false", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_use_faked_inos": "false", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "client_use_random_mds": "false", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "clog_to_graylog": "false", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "clog_to_graylog_port": "12201", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "clog_to_monitors": "default=true", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "clog_to_syslog": "false", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-22T03:11:12.069 INFO:teuthology.orchestra.run.smithi146.stdout: "clog_to_syslog_level": "info", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_addr": "-", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_network": "", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_network_interface": "", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "compressor_zlib_isal": "false", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "compressor_zlib_level": "5", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "compressor_zlib_winsize": "-15", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "compressor_zstd_level": "1", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "crimson_alien_op_num_threads": "6", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-22T03:11:12.070 INFO:teuthology.orchestra.run.smithi146.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "crimson_osd_stat_interval": "0", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "crimson_seastar_cpu_cores": "", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "crimson_seastar_num_threads": "0", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "crush_location": "", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "crush_location_hook": "", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "crush_location_hook_timeout": "10", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "daemonize": "false", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "daos_pool": "tank", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_alienstore": "0/5", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_allow_any_pool_priority": "false", 2024-09-22T03:11:12.071 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_asok": "1/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_asok_assert_abort": "false", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_asserts_on_shutdown": "true", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_auth": "20/20", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_bdev": "1/3", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_bluefs": "1/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_bluestore": "1/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_buffer": "0/1", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_ceph_exporter": "1/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_cephfs_mirror": "0/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_cephsqlite": "0/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_client": "0/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_compressor": "1/5", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_context": "0/1", 2024-09-22T03:11:12.072 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_crimson_interrupt": "0/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_crush": "1/1", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_crypto": "1/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_cyanstore": "0/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_deliberately_leak_memory": "false", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_disable_randomized_ping": "false", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_dpdk": "1/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_eventtrace": "1/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_filer": "0/1", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_filestore": "1/3", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_finisher": "1/1", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_fuse": "1/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_heartbeat_testing_span": "0", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_heartbeatmap": "1/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-22T03:11:12.073 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_javaclient": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_journal": "1/3", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_journaler": "0/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_kstore": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_lockdep": "0/1", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mclock": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mds": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mds_balancer": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mds_locker": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mds_log": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mds_log_expire": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mds_migrator": "1/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mds_quiesce": "3/5", 2024-09-22T03:11:12.074 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_memstore": "1/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mgr": "2/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mgrc": "1/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_mon": "20/20", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_monc": "0/10", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_ms": "1/1", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_none": "0/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_objclass": "0/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_objectcacher": "0/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_objecter": "0/1", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_optracker": "0/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_osd": "1/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_paxos": "20/20", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_perfcounter": "1/5", 2024-09-22T03:11:12.075 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_prioritycache": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rados": "0/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rbd": "0/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rbd_mirror": "0/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rbd_pwl": "0/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rbd_replay": "0/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_refs": "0/0", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_reserver": "1/1", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rgw": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rgw_access": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rgw_datacache": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rgw_dbstore": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rgw_flight": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rgw_sync": "1/5", 2024-09-22T03:11:12.076 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_rocksdb": "4/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_backref": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_cache": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_cleaner": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_device": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_epm": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_journal": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_lba": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_odata": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_omap": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_onode": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_t": "0/5", 2024-09-22T03:11:12.077 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_seastore_tm": "0/5", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_striper": "0/1", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_test": "0/5", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_throttle": "1/1", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_timer": "0/1", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_tp": "0/5", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "debug_trace": "1/5", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "device_failure_prediction_mode": "none", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "err_to_graylog": "false", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "err_to_journald": "false", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "err_to_stderr": "true", 2024-09-22T03:11:12.078 INFO:teuthology.orchestra.run.smithi146.stdout: "err_to_syslog": "false", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "event_tracing": "false", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "exporter_addr": "0.0.0.0", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "exporter_http_port": "9926", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "exporter_prio_limit": "5", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "exporter_sort_metrics": "true", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "exporter_stats_period": "5", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "fake_statfs_for_testing": "0", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "fatal_signal_handlers": "true", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "filer_max_purge_ops": "10", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "filer_max_truncate_ops": "128", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_apply_finisher_threads": "1", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_blackhole": "false", 2024-09-22T03:11:12.079 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_btrfs_clone_range": "true", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_btrfs_snap": "true", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_caller_concurrency": "10", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_collect_device_partition_information": "true", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_commit_timeout": "600.000000", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_debug_inject_read_err": "false", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_debug_omap_check": "false", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_debug_verify_split": "false", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_dump_file": "", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_fadvise": "true", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_fail_eio": "true", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_fd_cache_shards": "16", 2024-09-22T03:11:12.080 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_fd_cache_size": "128", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_fiemap": "false", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_fiemap_threshold": "4096", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_inject_stall": "0", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_journal_parallel": "false", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_journal_trailing": "false", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_journal_writeahead": "false", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_kill_at": "0", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-22T03:11:12.081 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattrs": "0", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_xattr_value_size": "0", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_merge_threshold": "-10", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_odsync_write": "false", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_omap_backend": "rocksdb", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_omap_backend_path": "", 2024-09-22T03:11:12.082 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_op_thread_timeout": "60", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_op_threads": "2", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_punch_hole": "false", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-22T03:11:12.083 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_queue_max_ops": "50", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_seek_data_hole": "false", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_sloppy_crc": "false", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_splice": "false", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_split_multiple": "2", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_split_rand_factor": "20", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_update_to": "1000", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-22T03:11:12.084 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_enable": "true", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_xfs_extsize": "false", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "filestore_zfs_snap": "false", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "fio_dir": "/tmp/fio", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "fsid": "39bf19db-1c34-4685-8c41-ee47ed066388", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_allow_other": "true", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_atomic_o_trunc": "true", 2024-09-22T03:11:12.085 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_big_writes": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_debug": "false", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_default_permissions": "false", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_disable_pagecache": "false", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_max_write": "0", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_multithreaded": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_require_active_mds": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_set_user_groups": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_splice_move": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_splice_read": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_splice_write": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "fuse_use_invalidate_cb": "true", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-22T03:11:12.086 INFO:teuthology.orchestra.run.smithi146.stdout: "gss_target_name": "ceph", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "heartbeat_file": "", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "heartbeat_inject_failure": "0", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "heartbeat_interval": "5", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "host": "smithi146", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-22T03:11:12.087 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "inject_early_sigterm": "false", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "jaeger_agent_port": "6799", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "jaeger_tracing_enable": "false", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_aio": "true", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_align_min_size": "65536", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_block_align": "true", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_block_size": "4096", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_dio": "true", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_discard": "false", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_force_aio": "false", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_ignore_corruption": "false", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_max_write_bytes": "10485760", 2024-09-22T03:11:12.088 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_max_write_entries": "100", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_replay_from": "0", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_write_header_frequency": "0", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journal_zero_on_create": "false", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journaler_prefetch_periods": "10", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journaler_prezero_periods": "5", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "journaler_write_head_interval": "15", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "keyfile": "", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-22T03:11:12.089 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_backend": "rocksdb", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_default_stripe_size": "65536", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_fsck_on_mount": "false", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_max_bytes": "67108864", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_max_ops": "512", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_nid_prealloc": "1024", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_onode_map_size": "1024", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_sync_submit_transaction": "false", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "kstore_sync_transaction": "false", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "librados_thread_count": "2", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "lockdep": "false", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "lockdep_force_backtrace": "false", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "log_coarse_timestamps": "true", 2024-09-22T03:11:12.090 INFO:teuthology.orchestra.run.smithi146.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_flush_on_exit": "false", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_graylog_host": "127.0.0.1", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_graylog_port": "12201", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_max_new": "1000", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_max_recent": "10000", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_stderr_prefix": "", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_stop_at_utilization": "0.970000", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_to_file": "true", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_to_graylog": "false", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_to_journald": "false", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_to_stderr": "false", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "log_to_syslog": "false", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "max_rotating_auth_attempts": "10", 2024-09-22T03:11:12.091 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_action_on_write_error": "1", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_alternate_name_max": "8192", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_asio_thread_count": "2", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_export_pin": "true", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_fragment_dirs": "true", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_fragment_interval": "5", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_interval": "10", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_max": "-1", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_max_until": "-1", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_merge_size": "50", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_midchunk": "0.300000", 2024-09-22T03:11:12.092 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_min_start": "0.200000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_minchunk": "0.001000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_mode": "0", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_need_max": "1.200000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_need_min": "0.800000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_overload_epochs": "2", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_split_bits": "3", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_split_size": "10000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_target_decay": "10.000000", 2024-09-22T03:11:12.093 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_beacon_grace": "15.000000", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_beacon_interval": "4.000000", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_mid": "0.700000", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_quiesce_delay": "0", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_release_free_interval": "10", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_reservation": "0.050000", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-22T03:11:12.094 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_trim_interval": "1", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache_trim_threshold": "262144", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_client_prealloc_inos": "1000", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_connect_bootstrapping": "false", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_damage_table_max_entries": "10000", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_debug_auth_pins": "false", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_debug_frag": "false", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_debug_scatterstat": "false", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_debug_subtrees": "false", 2024-09-22T03:11:12.095 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_decay_halflife": "5.000000", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_default_dir_hash": "2", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_defer_session_stale": "true", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_deny_all_reconnect": "false", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dir_keys_per_op": "16384", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dir_max_commit_size": "10", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dir_max_entries": "0", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dir_prefetch": "true", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dump_cache_on_map": "false", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_early_reply": "true", 2024-09-22T03:11:12.096 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_enable_op_tracker": "true", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_enforce_unique_name": "true", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_export_ephemeral_random": "true", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_health_summarize_threshold": "10", 2024-09-22T03:11:12.097 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_inject_health_dummy": "false", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_inject_migrator_session_race": "false", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_join_fs": "", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_journal_format": "1", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_create_at": "0", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_export_at": "0", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_import_at": "0", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_journal_at": "0", 2024-09-22T03:11:12.098 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_journal_expire_at": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_journal_replay_at": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_link_at": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_mdstable_at": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_openc_at": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_rename_at": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_kill_shutdown_at": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_event_large_threshold": "524288", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_events_per_segment": "1024", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_max_events": "-1", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_max_segments": "128", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_pause": "false", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_segment_size": "0", 2024-09-22T03:11:12.099 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_trim_threshold": "128", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log_warn_factor": "2.000000", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_caps_per_client": "1048576", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_completed_flushes": "100000", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_completed_requests": "100000", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_export_size": "20971520", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_file_recover": "32", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_purge_files": "64", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_purge_ops": "8192", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-22T03:11:12.100 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_max_snaps_per_dir": "100", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_metrics_update_interval": "2", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_min_caps_per_client": "100", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_min_caps_working_set": "10000", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_numa_node": "-1", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_op_complaint_time": "30.000000", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_op_history_duration": "600", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_op_history_size": "20", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_op_history_slow_op_size": "20", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_op_log_threshold": "5", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_ping_grace": "15", 2024-09-22T03:11:12.101 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_ping_interval": "5", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_recall_max_caps": "30000", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_recall_warning_threshold": "262144", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_replay_interval": "1.000000", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_root_ino_gid": "0", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_root_ino_uid": "0", 2024-09-22T03:11:12.102 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_shutdown_check": "0", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_skip_ino": "0", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_snap_max_uid": "4294967294", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_snap_min_uid": "0", 2024-09-22T03:11:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_snap_rstat": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_standby_replay_damaged": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_symlink_recovery": "true", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_thrash_exports": "0", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_thrash_fragments": "0", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_tick_interval": "5.000000", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_valgrind_exit": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_verify_backtrace": "1", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_verify_scatter": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_wipe_sessions": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool_debug": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-22T03:11:12.104 INFO:teuthology.orchestra.run.smithi146.stdout: "memstore_device_bytes": "1073741824", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "memstore_page_set": "false", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "memstore_page_size": "65536", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_client_bytes": "134217728", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_client_messages": "512", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_disabled_modules": "", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_enable_op_tracker": "true", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_max_pg_num_change": "128", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_mds_bytes": "134217728", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_mds_messages": "128", 2024-09-22T03:11:12.105 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_mon_bytes": "134217728", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_mon_messages": "128", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_op_history_duration": "600", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_op_history_size": "20", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_op_log_threshold": "5", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_osd_bytes": "536870912", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_osd_messages": "8192", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_pool": "true", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-22T03:11:12.106 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_standby_modules": "true", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_stats_period": "5", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_stats_threshold": "5", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_tick_period": "2", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_allow_pool_delete": "true", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_allow_pool_size_one": "true", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_auth_validate_all_caps": "true", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_bytes": "104857600", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_directed_command_retry": "2", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-22T03:11:12.107 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_hunt_parallel": "3", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_log_interval": "1.000000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_ping_interval": "10.000000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_client_target_rank": "-1", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_file_level": "debug", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_file": "true", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-22T03:11:12.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_journald": "false", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_compact_on_bootstrap": "false", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_compact_on_start": "false", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_compact_on_trim": "true", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_cpu_threads": "4", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_crush_min_required_version": "hammer", 2024-09-22T03:11:12.109 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_daemon_bytes": "419430400", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_data_avail_crit": "5", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_data_avail_warn": "5", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_data_size_warn": "16106127360", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_dump_json": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_dump_transactions": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_extra_checks": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_no_require_reef": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_no_require_squid": "false", 2024-09-22T03:11:12.110 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_down_mkfs_grace": "120", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_down_uptime_grace": "60", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_election_default_strategy": "1", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_election_timeout": "5.000000", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_elector_ping_divisor": "2", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_enable_op_tracker": "true", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_fake_pool_delete": "false", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_force_quorum_join": "false", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-22T03:11:12.111 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_globalid_prealloc": "10000", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_health_detail_to_clog": "false", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_health_log_update_period": "5", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_health_max_detail": "50", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_health_to_clog": "true", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_health_to_clog_interval": "600", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_host": "172.21.15.146", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_host_override": "", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_initial_members": "", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-22T03:11:12.112 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_lease": "5.000000", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_log_full_interval": "50", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_log_max": "10000", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_log_max_summary": "50", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_log_epochs": "500", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_osd": "10000", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_pg_per_osd": "10000", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_pool_pg_num": "65536", 2024-09-22T03:11:12.113 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mds_force_trim_to": "0", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mds_skip_sanity": "false", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_memory_autotune": "true", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_memory_target": "2147483648", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mgr_beacon_grace": "30", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mgr_digest_period": "5", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mgr_inactive_grace": "60", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_min_osdmap_epochs": "50", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_op_complaint_time": "30", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_op_history_duration": "600", 2024-09-22T03:11:12.114 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_op_history_size": "20", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_op_history_slow_op_size": "20", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_op_log_threshold": "5", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_auto_mark_in": "false", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_cache_size": "500", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-22T03:11:12.115 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_down_out_interval": "600", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_force_trim_to": "0", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-22T03:11:12.116 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_min_down_reporters": "2", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_report_timeout": "900", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-22T03:11:12.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pg_stuck_threshold": "60", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_probe_timeout": "2.000000", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_reweight_max_change": "0.050000", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_reweight_max_osds": "4", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-22T03:11:12.118 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_scrub_interval": "86400", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_scrub_max_keys": "100", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_scrub_timeout": "300", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_session_timeout": "300", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_smart_report_timeout": "5", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_stat_smooth_intervals": "6", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_stretch_pool_min_size": "2", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_stretch_pool_size": "4", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-22T03:11:12.119 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_sync_debug": "false", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_sync_provider_kill_at": "0", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_sync_requester_kill_at": "0", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_sync_timeout": "60.000000", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_target_pg_per_osd": "100", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_tick_interval": "5", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_timecheck_interval": "300.000000", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_use_min_delay_socket": "false", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_older_version_delay": "604800", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-22T03:11:12.120 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_misplaced": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_older_version": "true", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-22T03:11:12.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "monmap": "", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "motr_tracing_enabled": "false", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_op_threads": "3", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_cm": "false", 2024-09-22T03:11:12.122 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_device_name": "", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_dscp": "96", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_local_gid": "", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_port_num": "1", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_sl": "3", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_support_srq": "true", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_rdma_type": "ib", 2024-09-22T03:11:12.123 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_async_reap_threshold": "5", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_before_connect": "false", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_ipv4": "true", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_ipv6": "false", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_msgr1": "true", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_msgr2": "true", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_port_max": "7568", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_port_min": "6800", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_retry_count": "3", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_bind_retry_delay": "5", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_blackhole_client": "false", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_blackhole_mds": "false", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_blackhole_mgr": "false", 2024-09-22T03:11:12.124 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_blackhole_mon": "false", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_blackhole_osd": "false", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_client_mode": "crc", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_cluster_mode": "crc", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_cluster_type": "", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_compress_secure": "false", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_connection_idle_timeout": "900", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_connection_ready_timeout": "10", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_crc_data": "true", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_crc_header": "true", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_die_on_bad_msg": "false", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_die_on_bug": "true", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_die_on_old_message": "true", 2024-09-22T03:11:12.125 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_die_on_skipped_message": "false", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_coremask": "0xF", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_enable_tso": "true", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_hugepages": "", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_lro": "true", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_memory_channel": "4", 2024-09-22T03:11:12.126 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_pmd": "", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_port_id": "0", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_dump_on_send": "false", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_initial_backoff": "0.200000", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_inject_delay_max": "1.000000", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_inject_delay_type": "", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_inject_network_congestion": "0", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_inject_socket_failures": "0", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_learn_addr_from_peer": "true", 2024-09-22T03:11:12.127 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_max_accept_failures": "4", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_max_backoff": "15.000000", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_mon_client_mode": "crc", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_mon_cluster_mode": "crc", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_mon_service_mode": "crc", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_osd_compress_min_size": "1024", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_osd_compress_mode": "none", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_pq_min_cost": "65536", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_public_type": "", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_service_mode": "crc", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_tcp_listen_backlog": "512", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_tcp_nodelay": "true", 2024-09-22T03:11:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_tcp_rcvbuf": "0", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "ms_type": "async+posix", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "no_config_file": "false", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_completion_locks_per_session": "32", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_inflight_ops": "1024", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_tick_interval": "5.000000", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter_timeout": "10.000000", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objectstore_blackhole": "false", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-22T03:11:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: "openssl_engine_opts": "", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_agent_delay_time": "5.000000", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_agent_hist_halflife": "1000", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_agent_max_low_ops": "2", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_agent_max_ops": "4", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_agent_slop": "0.020000", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_asio_thread_count": "2", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_async_recovery_min_cost": "100", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-22T03:11:12.130 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_backfill_scan_max": "512", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_backfill_scan_min": "64", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_backoff_on_degraded": "false", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_backoff_on_peering": "false", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_backoff_on_unfound": "true", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_beacon_report_interval": "300", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bench_duration": "30", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bench_max_block_size": "67108864", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_blkin_trace_all": "false", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-22T03:11:12.131 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_check_for_log_corruption": "false", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_class_update_on_start": "true", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_client_message_cap": "256", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_client_message_size_cap": "524288000", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_client_op_priority": "63", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_client_watch_timeout": "30", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_command_max_records": "256", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_compact_on_start": "false", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-22T03:11:12.132 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_crush_update_on_start": "true", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_crush_update_weight_set": "true", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_feed_pullee": "-1", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_misdirected_ops": "false", 2024-09-22T03:11:12.133 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_no_acting_change": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_no_purge_strays": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_op_order": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_shutdown": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_trim_objects": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_verify_snaps": "false", 2024-09-22T03:11:12.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_deep_scrub_keys": "1024", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_deep_scrub_stride": "524288", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_default_notify_timeout": "30", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_delete_sleep": "0.000000", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_discard_disconnected_ops": "true", 2024-09-22T03:11:12.135 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_enable_op_tracker": "true", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_fast_info": "true", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_fast_shutdown": "true", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_function_tracing": "false", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_heartbeat_grace": "20", 2024-09-22T03:11:12.136 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_heartbeat_interval": "6", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_heartbeat_min_peers": "10", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_heartbeat_min_size": "2000", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_heartbeat_stale": "600", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_hit_set_max_size": "100000", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_hit_set_min_size": "1000", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_journal_size": "5120", 2024-09-22T03:11:12.137 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_kill_backfill_at": "0", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_map_cache_size": "50", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_map_dedup": "true", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_map_message_max": "40", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_map_share_max_epochs": "40", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_attr_name_len": "100", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_attr_size": "0", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_backfills": "1", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_markdown_count": "5", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_markdown_period": "600", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_object_name_len": "2048", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_object_namespace_len": "256", 2024-09-22T03:11:12.138 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_object_size": "134217728", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_pg_blocked_by": "16", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_pg_log_entries": "10000", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_pgls": "1024", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_push_cost": "8388608", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_push_objects": "10", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_scrubs": "3", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_trimming_pgs": "2", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_write_op_reply_len": "64", 2024-09-22T03:11:12.139 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_max_write_size": "90", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_profile": "balanced", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-22T03:11:12.140 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_memory_base": "805306368", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_memory_cache_min": "134217728", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_memory_target": "4294967296", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_memory_target_autotune": "false", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-22T03:11:12.141 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_min_pg_log_entries": "250", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_min_recovery_priority": "0", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mon_report_interval": "5", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_num_cache_shards": "32", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_num_op_tracker_shard": "32", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_numa_auto_affinity": "true", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_numa_node": "-1", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_numa_prefer_iface": "true", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_objecter_finishers": "1", 2024-09-22T03:11:12.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_objectstore": "bluestore", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_objectstore_fuse": "false", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_objectstore_tracing": "false", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_complaint_time": "30.000000", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_history_duration": "600", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_history_size": "20", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_history_slow_op_size": "20", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_log_threshold": "5", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_num_shards": "0", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_num_shards_hdd": "5", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_num_shards_ssd": "8", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-22T03:11:12.143 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_pq_min_cost": "65536", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_queue_cut_off": "high", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_op_thread_timeout": "15", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_open_classes_on_start": "true", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_os_flags": "0", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_peering_op_priority": "255", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_delete_cost": "1048576", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_delete_priority": "5", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_log_trim_max": "10000", 2024-09-22T03:11:12.144 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_log_trim_min": "100", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_crimson": "true", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-22T03:11:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_flags": "0", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_min_size": "2", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_pg_num": "32", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_pgp_num": "0", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_read_ratio": "70", 2024-09-22T03:11:12.146 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_size": "3", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_default_type": "replicated", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_push_per_object_cost": "1000", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recover_clone_overlap": "true", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_cost": "20971520", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_max_active": "0", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-22T03:11:12.147 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_max_single_start": "1", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_op_priority": "3", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_priority": "5", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_sleep": "0.000000", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_repair_during_recovery": "false", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_requested_scrub_priority": "120", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_auto_repair": "false", 2024-09-22T03:11:12.148 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_begin_hour": "0", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_begin_week_day": "0", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_chunk_max": "25", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_chunk_min": "5", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_cost": "52428800", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_during_recovery": "false", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_end_hour": "0", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_end_week_day": "0", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_event_cost": "4096", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_invalid_stats": "true", 2024-09-22T03:11:12.149 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_max_preemptions": "5", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_priority": "5", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_scrub_sleep": "0.000000", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_skip_data_digest": "false", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_smart_report_timeout": "5", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_snap_trim_cost": "1048576", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_snap_trim_priority": "5", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-22T03:11:12.150 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_target_transaction_size": "30", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-22T03:11:12.151 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_tracing": "false", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_use_stale_snap": "false", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "osdc_blkin_trace_all": "false", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_kill_at": "0", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_max_join_drift": "10", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_min": "500", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_min_wait": "0.050000", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_propose_interval": "1.000000", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_service_trim_max": "500", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-22T03:11:12.152 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_service_trim_min": "10", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_stash_full_interval": "25", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_trim_max": "500", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "paxos_trim_min": "250", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "perf": "true", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "public_addr": "-", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "public_addrv": "", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "public_bind_addr": "-", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "public_network": "", 2024-09-22T03:11:12.153 INFO:teuthology.orchestra.run.smithi146.stdout: "public_network_interface": "", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "qat_compressor_busy_polling": "false", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "qat_compressor_enabled": "false", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "qat_compressor_session_max_number": "256", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rados_mon_op_timeout": "0", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rados_osd_op_timeout": "0", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rados_tracing": "false", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_atime_update_interval": "60", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_balance_parent_reads": "false", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_balance_snap_reads": "false", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_blkin_trace_all": "false", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-22T03:11:12.154 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache": "true", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_policy": "writearound", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_size": "33554432", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_clone_copy_on_read": "false", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_compression_hint": "none", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_concurrent_management_ops": "10", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-22T03:11:12.155 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_clone_format": "auto", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_data_pool": "", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_features": "61", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_format": "2", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_map_options": "", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_order": "22", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_pool": "rbd", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_stripe_count": "0", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_default_stripe_unit": "0", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_enable_alloc_hint": "true", 2024-09-22T03:11:12.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_io_scheduler": "simple", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_order": "24", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_pool": "", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_journal_splay_width": "4", 2024-09-22T03:11:12.157 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_localize_parent_reads": "false", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_localize_snap_reads": "false", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-22T03:11:12.158 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-22T03:11:12.159 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_mtime_update_interval": "60", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_non_blocking_aio": "true", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_op_thread_timeout": "60", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_op_threads": "1", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_parent_cache_enabled": "false", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-22T03:11:12.160 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_plugins": "", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_bps_burst": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_bps_limit": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_exclude_ops": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_iops_burst": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_iops_limit": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-22T03:11:12.161 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_read_from_replica_policy": "default", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-22T03:11:12.162 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_skip_partial_discard": "true", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_tracing": "false", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_validate_names": "true", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rbd_validate_pool": "true", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "restapi_base_url": "", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "restapi_log_level": "", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_acl_grants_max_num": "100", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_admin_entry": "admin", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_backend_store": "rados", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_barbican_url": "", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_beast_enable_async": "true", 2024-09-22T03:11:12.163 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_counters_cache": "false", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_cache_enabled": "true", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_cache_expiry_interval": "900", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_cache_lru_size": "25000", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_config_store": "rados", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_content_length_compat": "false", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_copy_obj_progress": "true", 2024-09-22T03:11:12.164 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_cors_rules_max_num": "100", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_cross_domain_policy": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_addr": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_password": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_kmip_username": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_require_ssl": "true", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-22T03:11:12.165 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_addr": "", 2024-09-22T03:11:12.166 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_auth": "token", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_namespace": "", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_prefix": "", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_token_file": "", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_curl_buffersize": "524288", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_curl_low_speed_time": "300", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-22T03:11:12.167 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_d4n_port": "6379", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data_log_changes_size": "1000", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data_log_num_shards": "128", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data_log_window": "30", 2024-09-22T03:11:12.168 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-22T03:11:12.169 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dns_name": "", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dns_s3website_name": "", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_dynamic_resharding": "true", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enable_gc_threads": "true", 2024-09-22T03:11:12.170 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enable_lc_threads": "true", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enable_ops_log": "false", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enable_quota_threads": "true", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enable_static_website": "false", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enable_usage_log": "false", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_enforce_swift_acls": "true", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_exit_timeout_secs": "120", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_expose_bucket": "false", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_extended_http_attrs": "", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_filter": "none", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_frontends": "beast port=7480", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_max_deferred": "50", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-22T03:11:12.171 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_max_objs": "32", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_gc_processor_period": "3600", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_init_timeout": "300", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_kafka_connection_idle": "300", 2024-09-22T03:11:12.172 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_domain": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_password": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_password_path": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_project": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_tenant": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_token": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_token_path": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_admin_user": "", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_api_version": "2", 2024-09-22T03:11:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_barbican_domain": "", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_barbican_password": "", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_barbican_project": "", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_barbican_user": "", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_url": "", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lc_debug_interval": "-1", 2024-09-22T03:11:12.174 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lc_lock_max_time": "90", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lc_max_objs": "32", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lc_max_rules": "1000", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lc_max_worker": "3", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lc_thread_delay": "0", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ldap_dnattr": "uid", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ldap_searchfilter": "", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ldap_uri": "", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-22T03:11:12.175 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_log_http_headers": "", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_log_object_name_utc": "false", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_attr_name_len": "0", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_attr_size": "0", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_chunk_size": "4194304", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_listing_results": "1000", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_notify_retries": "10", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-22T03:11:12.176 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_put_param_size": "1048576", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_put_size": "5368709120", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_max_slo_entries": "1000", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_md_log_max_shards": "64", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_mp_lock_max_time": "600", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-22T03:11:12.177 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_max_gc": "300", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_num_async_rados_threads": "32", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_num_control_oids": "8", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_numa_node": "-1", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-22T03:11:12.178 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_objexp_chunk_size": "100", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_objexp_gc_interval": "600", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_op_thread_timeout": "600", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_op_tracing": "false", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_opa_token": "", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_opa_url": "", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_opa_verify_ssl": "true", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ops_log_rados": "false", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_ops_log_socket_path": "", 2024-09-22T03:11:12.179 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_period_push_interval": "2.000000", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_posix_cache_lanes": "3", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_posix_cache_partitions": "3", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_print_continue": "true", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-22T03:11:12.180 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_rados_tracing": "false", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_realm": "", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_realm_id": "", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_region": "", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_request_uri": "", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_reshard_batch_size": "64", 2024-09-22T03:11:12.181 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_reshard_max_aio": "128", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_reshard_num_logs": "16", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_reshard_thread_interval": "600", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_resolve_cname": "false", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_run_sync_thread": "true", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-22T03:11:12.182 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_scheduler_type": "throttler", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_script_uri": "", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_service_provider_name": "", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sts_client_id": "", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sts_client_secret": "", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sts_entry": "sts", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sts_key": "sts", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sts_min_session_duration": "900", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sts_token_introspection_url": "", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_account_in_url": "false", 2024-09-22T03:11:12.183 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_auth_entry": "auth", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_auth_url": "", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_custom_header": "", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_need_stats": "true", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_tenant_name": "", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_token_expiration": "86400", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_url": "", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_url_prefix": "swift", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_lease_period": "120", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-22T03:11:12.184 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_thread_pool_size": "512", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_comment": "", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_createby": "", 2024-09-22T03:11:12.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_encoding": "", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_flag": "false", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_origin": "", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_torrent_tracker": "", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_trust_forwarded_https": "false", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_usage_max_shards": "32", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_usage_max_user_shards": "1", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_use_opa_authz": "false", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_counters_cache": "false", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-22T03:11:12.186 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_max_buckets": "1000", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_policies_max_num": "100", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_user_unique_email": "true", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_verify_ssl": "true", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_zone": "", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_zone_id": "", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_zonegroup": "", 2024-09-22T03:11:12.187 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_zonegroup_id": "", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rgwlc_auto_session_clear": "true", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_block_size": "4096", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cache_size": "536870912", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-22T03:11:12.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_index_type": "binary_search", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_partition_filters": "false", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_perf": "false", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "run_dir": "/var/run/ceph", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_block_create": "true", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_cache_lru_size": "67108864", 2024-09-22T03:11:12.189 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_default_max_object_size": "16777216", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_device_size": "53687091200", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_full_integrity_check": "false", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_journal_batch_capacity": "16", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_main_device_type": "SSD", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-22T03:11:12.190 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "seastore_segment_size": "67108864", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "set_keepcaps": "false", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "setgroup": "", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "setuser": "", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "setuser_match_path": "", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "thp": "false", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "threadpool_default_timeout": "60", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout: "throttler_perf_counter": "true" 2024-09-22T03:11:12.191 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:11:12.193 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-22T03:11:12.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:12.295+0000 7f7dbe517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:12.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:12.298+0000 7f7dbe517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:12.313 INFO:teuthology.orchestra.run.smithi146.stdout:{} 2024-09-22T03:11:12.314 INFO:teuthology.orchestra.run.smithi146.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T03:11:12.324 DEBUG:teuthology.orchestra.run.smithi148:> type ceph-objectstore-tool 2024-09-22T03:11:12.342 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T03:11:12.343 DEBUG:teuthology.orchestra.run.smithi146:> type ceph-objectstore-tool 2024-09-22T03:11:12.357 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T03:11:12.357 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-22T03:11:12.367 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:11:12.368 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:11:12.368 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-22T03:11:12.368 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-22T03:11:12.369 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-22T03:11:12.369 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:11:12.369 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:11:12.369 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3] 2024-09-22T03:11:12.369 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:11:12.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:11:12.369 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-22T03:11:12.370 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:11:12.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:12.502+0000 7f8d4646c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:12.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:12.507+0000 7f8d4646c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:12.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:12.501+0000 7fb19c26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:12.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:12.511+0000 7fb19c26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:13.392 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_0' created 2024-09-22T03:11:13.407 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-22T03:11:13.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:13.546+0000 7fb600997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:13.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:13.556+0000 7fb600997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:14.392 INFO:teuthology.orchestra.run.smithi146.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-22T03:11:14.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:11:15.412 DEBUG:teuthology.orchestra.run.smithi146:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-22T03:11:15.415 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:11:15.415 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:11:15.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:hints = 1 2024-09-22T03:11:15.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:11:15.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object prefix: benchmark_data_smithi146_29996 2024-09-22T03:11:15.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:11:15.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:11:16.476 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 1 16 25 9 0.562427 0.5625 0.0110166 0.0321614 2024-09-22T03:11:17.476 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 2 16 25 9 0.281216 0 - 0.0321614 2024-09-22T03:11:18.471 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:11:18.476 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 3 16 25 9 0.187477 0 - 0.0321614 2024-09-22T03:11:19.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 4 16 25 9 0.140608 0 - 0.0321614 2024-09-22T03:11:19.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:11:20.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 5 16 25 9 0.112487 0 - 0.0321614 2024-09-22T03:11:21.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 6 16 25 9 0.0937387 0 - 0.0321614 2024-09-22T03:11:22.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 7 16 25 9 0.0803472 0 - 0.0321614 2024-09-22T03:11:23.472 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-22T03:11:23.473 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:11:23.473 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:11:23.473 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:11:23.476 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:11:23.476 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:11:23.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 8 16 25 9 0.0703036 0 - 0.0321614 2024-09-22T03:11:23.587 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:11:23.590 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:11:23.590 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:11:24.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 9 16 25 9 0.0624921 0 - 0.0321614 2024-09-22T03:11:25.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 10 16 25 9 0.0562428 0 - 0.0321614 2024-09-22T03:11:26.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 11 16 25 9 0.0511298 0 - 0.0321614 2024-09-22T03:11:27.411 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:11:27.477 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 12 16 25 9 0.0468692 0 - 0.0321614 2024-09-22T03:11:27.510 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:11:27.511 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:11:28.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 13 16 25 9 0.043264 0 - 0.0321614 2024-09-22T03:11:28.591 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:11:28.693 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:11:28.693 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [] 2024-09-22T03:11:28.693 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:11:28.702 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:11:28.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:28.801+0000 7f40740fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:28.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:28.810+0000 7f40740fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:29.416 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:11:29.437 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:11:29.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 14 16 25 9 0.0401739 0 - 0.0321614 2024-09-22T03:11:29.512 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:11:29.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:29.588+0000 7fb7d78b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:29.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:29.603+0000 7fb7d78b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:29.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:29.655+0000 7fee9c557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:29.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:29.664+0000 7fee9c557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:30.419 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:11:30.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:11:30.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 15 16 361 345 1.43734 1.5 0.0467723 0.694568 2024-09-22T03:11:30.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:30.588+0000 7fb702d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:30.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:30.604+0000 7fb702d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:31.420 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:11:31.441 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:11:31.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 16 16 790 774 3.0231 26.8125 0.0242213 0.330405 2024-09-22T03:11:31.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:31.641+0000 7f8f03a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:31.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:31.658+0000 7f8f03a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:31.810 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:11:31.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:31.972+0000 7f2a2842c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:31.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:31.981+0000 7f2a271ca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:32.425 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:11:32.447 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:11:32.447 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:32.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 17 16 1218 1202 4.41862 26.75 0.0356104 0.225846 2024-09-22T03:11:32.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:32.598+0000 7f89d7a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:32.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:32.609+0000 7f89d7a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:32.976 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:32.976 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:32.995 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-22T03:11:32.426171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20054579,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":436,"num_write_kb":20904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":60,"acting":60,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":110060,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377377300,"statfs":{"total":386547056640,"available":386434355200,"internally_reserved":0,"allocated":2785280,"data_stored":34052259,"data_compressed":1614189,"data_compressed_allocated":2129920,"data_compressed_original":33832960,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016362"},"pg_stats":[{"pgid":"3.f","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.133267+0000","last_change":"2024-09-22T03:11:29.505342+0000","last_active":"2024-09-22T03:11:32.133267+0000","last_peered":"2024-09-22T03:11:32.133267+0000","last_clean":"2024-09-22T03:11:32.133267+0000","last_became_active":"2024-09-22T03:11:29.496134+0000","last_became_peered":"2024-09-22T03:11:29.496134+0000","last_unstale":"2024-09-22T03:11:32.133267+0000","last_undegraded":"2024-09-22T03:11:32.133267+0000","last_fullsized":"2024-09-22T03:11:32.133267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.477856+0000","last_change":"2024-09-22T03:11:27.477856+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.477856+0000","last_undegraded":"2024-09-22T03:11:27.477856+0000","last_fullsized":"2024-09-22T03:11:27.477856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'73","reported_seq":101,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.209856+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:11:32.209856+0000","last_peered":"2024-09-22T03:11:32.209856+0000","last_clean":"2024-09-22T03:11:32.209856+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:11:32.209856+0000","last_undegraded":"2024-09-22T03:11:32.209856+0000","last_fullsized":"2024-09-22T03:11:32.209856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.431112+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:11:27.431112+0000","last_peered":"2024-09-22T03:11:27.431112+0000","last_clean":"2024-09-22T03:11:27.431112+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:11:27.431112+0000","last_undegraded":"2024-09-22T03:11:27.431112+0000","last_fullsized":"2024-09-22T03:11:27.431112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.132148+0000","last_change":"2024-09-22T03:11:29.503432+0000","last_active":"2024-09-22T03:11:32.132148+0000","last_peered":"2024-09-22T03:11:32.132148+0000","last_clean":"2024-09-22T03:11:32.132148+0000","last_became_active":"2024-09-22T03:11:29.492577+0000","last_became_peered":"2024-09-22T03:11:29.492577+0000","last_unstale":"2024-09-22T03:11:32.132148+0000","last_undegraded":"2024-09-22T03:11:32.132148+0000","last_fullsized":"2024-09-22T03:11:32.132148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"22'78","reported_seq":98,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.233358+0000","last_change":"2024-09-22T03:11:29.523421+0000","last_active":"2024-09-22T03:11:32.233358+0000","last_peered":"2024-09-22T03:11:32.233358+0000","last_clean":"2024-09-22T03:11:32.233358+0000","last_became_active":"2024-09-22T03:11:29.515869+0000","last_became_peered":"2024-09-22T03:11:29.515869+0000","last_unstale":"2024-09-22T03:11:32.233358+0000","last_undegraded":"2024-09-22T03:11:32.233358+0000","last_fullsized":"2024-09-22T03:11:32.233358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.470510+0000","last_change":"2024-09-22T03:11:27.470510+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.470510+0000","last_undegraded":"2024-09-22T03:11:27.470510+0000","last_fullsized":"2024-09-22T03:11:27.470510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.415904+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:27.415904+0000","last_peered":"2024-09-22T03:11:27.415904+0000","last_clean":"2024-09-22T03:11:27.415904+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:27.415904+0000","last_undegraded":"2024-09-22T03:11:27.415904+0000","last_fullsized":"2024-09-22T03:11:27.415904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.444471+0000","last_change":"2024-09-22T03:11:30.445307+0000","last_active":"2024-09-22T03:11:31.444471+0000","last_peered":"2024-09-22T03:11:31.444471+0000","last_clean":"2024-09-22T03:11:31.444471+0000","last_became_active":"2024-09-22T03:11:29.457496+0000","last_became_peered":"2024-09-22T03:11:29.457496+0000","last_unstale":"2024-09-22T03:11:31.444471+0000","last_undegraded":"2024-09-22T03:11:31.444471+0000","last_fullsized":"2024-09-22T03:11:31.444471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"17'3","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.434266+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:11:27.434266+0000","last_peered":"2024-09-22T03:11:27.434266+0000","last_clean":"2024-09-22T03:11:27.434266+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:11:27.434266+0000","last_undegraded":"2024-09-22T03:11:27.434266+0000","last_fullsized":"2024-09-22T03:11:27.434266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":28,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:15.407986+0000","last_change":"2024-09-22T03:11:27.420723+0000","last_active":"2024-09-22T03:11:27.420723+0000","last_peered":"2024-09-22T03:11:15.407986+0000","last_clean":"2024-09-22T03:11:15.407986+0000","last_became_active":"2024-09-22T03:11:06.312069+0000","last_became_peered":"2024-09-22T03:11:06.312069+0000","last_unstale":"2024-09-22T03:11:15.407986+0000","last_undegraded":"2024-09-22T03:11:15.407986+0000","last_fullsized":"2024-09-22T03:11:15.407986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":27,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:15.410215+0000","last_change":"2024-09-22T03:11:27.439105+0000","last_active":"2024-09-22T03:11:27.439105+0000","last_peered":"2024-09-22T03:11:15.410215+0000","last_clean":"2024-09-22T03:11:15.410215+0000","last_became_active":"2024-09-22T03:11:06.303791+0000","last_became_peered":"2024-09-22T03:11:06.303791+0000","last_unstale":"2024-09-22T03:11:15.410215+0000","last_undegraded":"2024-09-22T03:11:15.410215+0000","last_fullsized":"2024-09-22T03:11:15.410215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.448251+0000","last_change":"2024-09-22T03:11:29.502756+0000","last_active":"2024-09-22T03:11:31.448251+0000","last_peered":"2024-09-22T03:11:31.448251+0000","last_clean":"2024-09-22T03:11:31.448251+0000","last_became_active":"2024-09-22T03:11:29.498892+0000","last_became_peered":"2024-09-22T03:11:29.498892+0000","last_unstale":"2024-09-22T03:11:31.448251+0000","last_undegraded":"2024-09-22T03:11:31.448251+0000","last_fullsized":"2024-09-22T03:11:31.448251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:08.276778+0000","last_change":"2024-09-22T03:11:08.278389+0000","last_active":"2024-09-22T03:11:08.276778+0000","last_peered":"2024-09-22T03:11:08.276778+0000","last_clean":"2024-09-22T03:11:08.276778+0000","last_became_active":"2024-09-22T03:11:06.309361+0000","last_became_peered":"2024-09-22T03:11:06.309361+0000","last_unstale":"2024-09-22T03:11:27.408583+0000","last_undegraded":"2024-09-22T03:11:08.276778+0000","last_fullsized":"2024-09-22T03:11:08.276778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.432900+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:27.432900+0000","last_peered":"2024-09-22T03:11:27.432900+0000","last_clean":"2024-09-22T03:11:27.432900+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:27.432900+0000","last_undegraded":"2024-09-22T03:11:27.432900+0000","last_fullsized":"2024-09-22T03:11:27.432900+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"17'4","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.416131+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:11:27.416131+0000","last_peered":"2024-09-22T03:11:27.416131+0000","last_clean":"2024-09-22T03:11:27.416131+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:11:27.416131+0000","last_undegraded":"2024-09-22T03:11:27.416131+0000","last_fullsized":"2024-09-22T03:11:27.416131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:06.266491+0000","last_change":"2024-09-22T03:11:27.434769+0000","last_active":"2024-09-22T03:11:05.253234+0000","last_peered":"2024-09-22T03:11:05.253234+0000","last_clean":"2024-09-22T03:11:05.253234+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:06.266491+0000","last_undegraded":"2024-09-22T03:11:06.266491+0000","last_fullsized":"2024-09-22T03:11:06.266491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.454594+0000","last_change":"2024-09-22T03:11:27.454594+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.454594+0000","last_undegraded":"2024-09-22T03:11:27.454594+0000","last_fullsized":"2024-09-22T03:11:27.454594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:06.266485+0000","last_change":"2024-09-22T03:11:27.423555+0000","last_active":"2024-09-22T03:11:05.253234+0000","last_peered":"2024-09-22T03:11:05.253234+0000","last_clean":"2024-09-22T03:11:05.253234+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:06.266485+0000","last_undegraded":"2024-09-22T03:11:06.266485+0000","last_fullsized":"2024-09-22T03:11:06.266485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":19464192,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":19008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":51904512,"data_compressed":2463876,"data_compressed_allocated":3244032,"data_compressed_original":51904512,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":37,"acting":37,"num_store_stats":3},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":20,"acting":20,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27012,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344828,"statfs":{"total":96636764160,"available":96609103872,"internally_reserved":0,"allocated":184320,"data_stored":631079,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27072,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":245760,"data_stored":1221447,"data_compressed":57957,"data_compressed_allocated":77824,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705693,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28952,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94342888,"statfs":{"total":96636764160,"available":96607117312,"internally_reserved":0,"allocated":2170880,"data_stored":31578608,"data_compressed":1498275,"data_compressed_allocated":1974272,"data_compressed_original":31506432,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3170304,"data_stored":50724864,"data_compressed":2407906,"data_compressed_allocated":3170304,"data_compressed_original":50724864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":589824,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":589824,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:32.998 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:33.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:33.187+0000 7f8b21f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:33.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:33.199+0000 7f8b21f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:33.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 18 16 1632 1616 5.61049 25.875 0.0427101 0.177968 2024-09-22T03:11:33.547 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:33.547 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:33.565 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-22T03:11:33.228186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20054579,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":436,"num_write_kb":20904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":60,"acting":60,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":110060,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377377300,"statfs":{"total":386547056640,"available":386434355200,"internally_reserved":0,"allocated":2785280,"data_stored":34052259,"data_compressed":1614189,"data_compressed_allocated":2129920,"data_compressed_original":33832960,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000730"},"pg_stats":[{"pgid":"3.f","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.133267+0000","last_change":"2024-09-22T03:11:29.505342+0000","last_active":"2024-09-22T03:11:32.133267+0000","last_peered":"2024-09-22T03:11:32.133267+0000","last_clean":"2024-09-22T03:11:32.133267+0000","last_became_active":"2024-09-22T03:11:29.496134+0000","last_became_peered":"2024-09-22T03:11:29.496134+0000","last_unstale":"2024-09-22T03:11:32.133267+0000","last_undegraded":"2024-09-22T03:11:32.133267+0000","last_fullsized":"2024-09-22T03:11:32.133267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.477856+0000","last_change":"2024-09-22T03:11:27.477856+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.477856+0000","last_undegraded":"2024-09-22T03:11:27.477856+0000","last_fullsized":"2024-09-22T03:11:27.477856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'73","reported_seq":101,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.209856+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:11:32.209856+0000","last_peered":"2024-09-22T03:11:32.209856+0000","last_clean":"2024-09-22T03:11:32.209856+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:11:32.209856+0000","last_undegraded":"2024-09-22T03:11:32.209856+0000","last_fullsized":"2024-09-22T03:11:32.209856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.431112+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:11:27.431112+0000","last_peered":"2024-09-22T03:11:27.431112+0000","last_clean":"2024-09-22T03:11:27.431112+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:11:27.431112+0000","last_undegraded":"2024-09-22T03:11:27.431112+0000","last_fullsized":"2024-09-22T03:11:27.431112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.132148+0000","last_change":"2024-09-22T03:11:29.503432+0000","last_active":"2024-09-22T03:11:32.132148+0000","last_peered":"2024-09-22T03:11:32.132148+0000","last_clean":"2024-09-22T03:11:32.132148+0000","last_became_active":"2024-09-22T03:11:29.492577+0000","last_became_peered":"2024-09-22T03:11:29.492577+0000","last_unstale":"2024-09-22T03:11:32.132148+0000","last_undegraded":"2024-09-22T03:11:32.132148+0000","last_fullsized":"2024-09-22T03:11:32.132148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"22'78","reported_seq":98,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.233358+0000","last_change":"2024-09-22T03:11:29.523421+0000","last_active":"2024-09-22T03:11:32.233358+0000","last_peered":"2024-09-22T03:11:32.233358+0000","last_clean":"2024-09-22T03:11:32.233358+0000","last_became_active":"2024-09-22T03:11:29.515869+0000","last_became_peered":"2024-09-22T03:11:29.515869+0000","last_unstale":"2024-09-22T03:11:32.233358+0000","last_undegraded":"2024-09-22T03:11:32.233358+0000","last_fullsized":"2024-09-22T03:11:32.233358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.470510+0000","last_change":"2024-09-22T03:11:27.470510+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.470510+0000","last_undegraded":"2024-09-22T03:11:27.470510+0000","last_fullsized":"2024-09-22T03:11:27.470510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.415904+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:27.415904+0000","last_peered":"2024-09-22T03:11:27.415904+0000","last_clean":"2024-09-22T03:11:27.415904+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:27.415904+0000","last_undegraded":"2024-09-22T03:11:27.415904+0000","last_fullsized":"2024-09-22T03:11:27.415904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.444471+0000","last_change":"2024-09-22T03:11:30.445307+0000","last_active":"2024-09-22T03:11:31.444471+0000","last_peered":"2024-09-22T03:11:31.444471+0000","last_clean":"2024-09-22T03:11:31.444471+0000","last_became_active":"2024-09-22T03:11:29.457496+0000","last_became_peered":"2024-09-22T03:11:29.457496+0000","last_unstale":"2024-09-22T03:11:31.444471+0000","last_undegraded":"2024-09-22T03:11:31.444471+0000","last_fullsized":"2024-09-22T03:11:31.444471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"17'3","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.434266+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:11:27.434266+0000","last_peered":"2024-09-22T03:11:27.434266+0000","last_clean":"2024-09-22T03:11:27.434266+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:11:27.434266+0000","last_undegraded":"2024-09-22T03:11:27.434266+0000","last_fullsized":"2024-09-22T03:11:27.434266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":28,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:15.407986+0000","last_change":"2024-09-22T03:11:27.420723+0000","last_active":"2024-09-22T03:11:27.420723+0000","last_peered":"2024-09-22T03:11:15.407986+0000","last_clean":"2024-09-22T03:11:15.407986+0000","last_became_active":"2024-09-22T03:11:06.312069+0000","last_became_peered":"2024-09-22T03:11:06.312069+0000","last_unstale":"2024-09-22T03:11:15.407986+0000","last_undegraded":"2024-09-22T03:11:15.407986+0000","last_fullsized":"2024-09-22T03:11:15.407986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":27,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:15.410215+0000","last_change":"2024-09-22T03:11:27.439105+0000","last_active":"2024-09-22T03:11:27.439105+0000","last_peered":"2024-09-22T03:11:15.410215+0000","last_clean":"2024-09-22T03:11:15.410215+0000","last_became_active":"2024-09-22T03:11:06.303791+0000","last_became_peered":"2024-09-22T03:11:06.303791+0000","last_unstale":"2024-09-22T03:11:15.410215+0000","last_undegraded":"2024-09-22T03:11:15.410215+0000","last_fullsized":"2024-09-22T03:11:15.410215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.448251+0000","last_change":"2024-09-22T03:11:29.502756+0000","last_active":"2024-09-22T03:11:31.448251+0000","last_peered":"2024-09-22T03:11:31.448251+0000","last_clean":"2024-09-22T03:11:31.448251+0000","last_became_active":"2024-09-22T03:11:29.498892+0000","last_became_peered":"2024-09-22T03:11:29.498892+0000","last_unstale":"2024-09-22T03:11:31.448251+0000","last_undegraded":"2024-09-22T03:11:31.448251+0000","last_fullsized":"2024-09-22T03:11:31.448251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:08.276778+0000","last_change":"2024-09-22T03:11:08.278389+0000","last_active":"2024-09-22T03:11:08.276778+0000","last_peered":"2024-09-22T03:11:08.276778+0000","last_clean":"2024-09-22T03:11:08.276778+0000","last_became_active":"2024-09-22T03:11:06.309361+0000","last_became_peered":"2024-09-22T03:11:06.309361+0000","last_unstale":"2024-09-22T03:11:27.408583+0000","last_undegraded":"2024-09-22T03:11:08.276778+0000","last_fullsized":"2024-09-22T03:11:08.276778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.432900+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:27.432900+0000","last_peered":"2024-09-22T03:11:27.432900+0000","last_clean":"2024-09-22T03:11:27.432900+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:27.432900+0000","last_undegraded":"2024-09-22T03:11:27.432900+0000","last_fullsized":"2024-09-22T03:11:27.432900+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"17'4","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.416131+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:11:27.416131+0000","last_peered":"2024-09-22T03:11:27.416131+0000","last_clean":"2024-09-22T03:11:27.416131+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:11:27.416131+0000","last_undegraded":"2024-09-22T03:11:27.416131+0000","last_fullsized":"2024-09-22T03:11:27.416131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:06.266491+0000","last_change":"2024-09-22T03:11:27.434769+0000","last_active":"2024-09-22T03:11:05.253234+0000","last_peered":"2024-09-22T03:11:05.253234+0000","last_clean":"2024-09-22T03:11:05.253234+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:06.266491+0000","last_undegraded":"2024-09-22T03:11:06.266491+0000","last_fullsized":"2024-09-22T03:11:06.266491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.454594+0000","last_change":"2024-09-22T03:11:27.454594+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.454594+0000","last_undegraded":"2024-09-22T03:11:27.454594+0000","last_fullsized":"2024-09-22T03:11:27.454594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:06.266485+0000","last_change":"2024-09-22T03:11:27.423555+0000","last_active":"2024-09-22T03:11:05.253234+0000","last_peered":"2024-09-22T03:11:05.253234+0000","last_clean":"2024-09-22T03:11:05.253234+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:06.266485+0000","last_undegraded":"2024-09-22T03:11:06.266485+0000","last_fullsized":"2024-09-22T03:11:06.266485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":19464192,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":19008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":51904512,"data_compressed":2463876,"data_compressed_allocated":3244032,"data_compressed_original":51904512,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":37,"acting":37,"num_store_stats":3},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":20,"acting":20,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27012,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344828,"statfs":{"total":96636764160,"available":96609103872,"internally_reserved":0,"allocated":184320,"data_stored":631079,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27072,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":245760,"data_stored":1221447,"data_compressed":57957,"data_compressed_allocated":77824,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705693,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28952,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94342888,"statfs":{"total":96636764160,"available":96607117312,"internally_reserved":0,"allocated":2170880,"data_stored":31578608,"data_compressed":1498275,"data_compressed_allocated":1974272,"data_compressed_original":31506432,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3170304,"data_stored":50724864,"data_compressed":2407906,"data_compressed_allocated":3170304,"data_compressed_original":50724864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":589824,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":589824,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:33.566 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:11:33.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:33.711+0000 7f0d35f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:33.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:33.723+0000 7f0d35f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:34.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:11:34.229 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:34.229 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":39,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":23,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974688,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":11},{"state_name":"creating+peering","count":8},{"state_name":"peering","count":4},{"state_name":"unknown","count":1},{"state_name":"stale+active+clean","count":1}],"num_pgs":25,"num_pools":3,"num_objects":301,"data_bytes":20054579,"bytes_used":112701440,"bytes_avail":386434355200,"bytes_total":386547056640,"unknown_pgs_ratio":0.039999999105930328,"inactive_pgs_ratio":0.47999998927116394,"write_bytes_sec":4717730,"read_op_per_sec":0,"write_op_per_sec":71},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-22T03:10:55.011428+0000","services":{}},"progress_events":{}} 2024-09-22T03:11:34.252 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-22T03:11:34.252 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:34.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:34.344+0000 7fafe20d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:34.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:34.359+0000 7fafe20d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:34.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:34.409+0000 7f813786c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:34.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:34.424+0000 7f813786c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:34.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 19 16 2027 2011 6.61441 24.6875 0.0746223 0.150988 2024-09-22T03:11:34.739 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:34.740 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:34.761 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-22T03:11:33.228186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20054579,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":436,"num_write_kb":20904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":60,"acting":60,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":110060,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377377300,"statfs":{"total":386547056640,"available":386434355200,"internally_reserved":0,"allocated":2785280,"data_stored":34052259,"data_compressed":1614189,"data_compressed_allocated":2129920,"data_compressed_original":33832960,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000730"},"pg_stats":[{"pgid":"3.f","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.133267+0000","last_change":"2024-09-22T03:11:29.505342+0000","last_active":"2024-09-22T03:11:32.133267+0000","last_peered":"2024-09-22T03:11:32.133267+0000","last_clean":"2024-09-22T03:11:32.133267+0000","last_became_active":"2024-09-22T03:11:29.496134+0000","last_became_peered":"2024-09-22T03:11:29.496134+0000","last_unstale":"2024-09-22T03:11:32.133267+0000","last_undegraded":"2024-09-22T03:11:32.133267+0000","last_fullsized":"2024-09-22T03:11:32.133267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.477856+0000","last_change":"2024-09-22T03:11:27.477856+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.477856+0000","last_undegraded":"2024-09-22T03:11:27.477856+0000","last_fullsized":"2024-09-22T03:11:27.477856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'73","reported_seq":101,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.209856+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:11:32.209856+0000","last_peered":"2024-09-22T03:11:32.209856+0000","last_clean":"2024-09-22T03:11:32.209856+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:11:32.209856+0000","last_undegraded":"2024-09-22T03:11:32.209856+0000","last_fullsized":"2024-09-22T03:11:32.209856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.431112+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:11:27.431112+0000","last_peered":"2024-09-22T03:11:27.431112+0000","last_clean":"2024-09-22T03:11:27.431112+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:11:27.431112+0000","last_undegraded":"2024-09-22T03:11:27.431112+0000","last_fullsized":"2024-09-22T03:11:27.431112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.132148+0000","last_change":"2024-09-22T03:11:29.503432+0000","last_active":"2024-09-22T03:11:32.132148+0000","last_peered":"2024-09-22T03:11:32.132148+0000","last_clean":"2024-09-22T03:11:32.132148+0000","last_became_active":"2024-09-22T03:11:29.492577+0000","last_became_peered":"2024-09-22T03:11:29.492577+0000","last_unstale":"2024-09-22T03:11:32.132148+0000","last_undegraded":"2024-09-22T03:11:32.132148+0000","last_fullsized":"2024-09-22T03:11:32.132148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"22'78","reported_seq":98,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.233358+0000","last_change":"2024-09-22T03:11:29.523421+0000","last_active":"2024-09-22T03:11:32.233358+0000","last_peered":"2024-09-22T03:11:32.233358+0000","last_clean":"2024-09-22T03:11:32.233358+0000","last_became_active":"2024-09-22T03:11:29.515869+0000","last_became_peered":"2024-09-22T03:11:29.515869+0000","last_unstale":"2024-09-22T03:11:32.233358+0000","last_undegraded":"2024-09-22T03:11:32.233358+0000","last_fullsized":"2024-09-22T03:11:32.233358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.470510+0000","last_change":"2024-09-22T03:11:27.470510+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.470510+0000","last_undegraded":"2024-09-22T03:11:27.470510+0000","last_fullsized":"2024-09-22T03:11:27.470510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.415904+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:27.415904+0000","last_peered":"2024-09-22T03:11:27.415904+0000","last_clean":"2024-09-22T03:11:27.415904+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:27.415904+0000","last_undegraded":"2024-09-22T03:11:27.415904+0000","last_fullsized":"2024-09-22T03:11:27.415904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.444471+0000","last_change":"2024-09-22T03:11:30.445307+0000","last_active":"2024-09-22T03:11:31.444471+0000","last_peered":"2024-09-22T03:11:31.444471+0000","last_clean":"2024-09-22T03:11:31.444471+0000","last_became_active":"2024-09-22T03:11:29.457496+0000","last_became_peered":"2024-09-22T03:11:29.457496+0000","last_unstale":"2024-09-22T03:11:31.444471+0000","last_undegraded":"2024-09-22T03:11:31.444471+0000","last_fullsized":"2024-09-22T03:11:31.444471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"17'3","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.434266+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:11:27.434266+0000","last_peered":"2024-09-22T03:11:27.434266+0000","last_clean":"2024-09-22T03:11:27.434266+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:11:27.434266+0000","last_undegraded":"2024-09-22T03:11:27.434266+0000","last_fullsized":"2024-09-22T03:11:27.434266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":28,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:15.407986+0000","last_change":"2024-09-22T03:11:27.420723+0000","last_active":"2024-09-22T03:11:27.420723+0000","last_peered":"2024-09-22T03:11:15.407986+0000","last_clean":"2024-09-22T03:11:15.407986+0000","last_became_active":"2024-09-22T03:11:06.312069+0000","last_became_peered":"2024-09-22T03:11:06.312069+0000","last_unstale":"2024-09-22T03:11:15.407986+0000","last_undegraded":"2024-09-22T03:11:15.407986+0000","last_fullsized":"2024-09-22T03:11:15.407986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":27,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:15.410215+0000","last_change":"2024-09-22T03:11:27.439105+0000","last_active":"2024-09-22T03:11:27.439105+0000","last_peered":"2024-09-22T03:11:15.410215+0000","last_clean":"2024-09-22T03:11:15.410215+0000","last_became_active":"2024-09-22T03:11:06.303791+0000","last_became_peered":"2024-09-22T03:11:06.303791+0000","last_unstale":"2024-09-22T03:11:15.410215+0000","last_undegraded":"2024-09-22T03:11:15.410215+0000","last_fullsized":"2024-09-22T03:11:15.410215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.448251+0000","last_change":"2024-09-22T03:11:29.502756+0000","last_active":"2024-09-22T03:11:31.448251+0000","last_peered":"2024-09-22T03:11:31.448251+0000","last_clean":"2024-09-22T03:11:31.448251+0000","last_became_active":"2024-09-22T03:11:29.498892+0000","last_became_peered":"2024-09-22T03:11:29.498892+0000","last_unstale":"2024-09-22T03:11:31.448251+0000","last_undegraded":"2024-09-22T03:11:31.448251+0000","last_fullsized":"2024-09-22T03:11:31.448251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":14,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:08.276778+0000","last_change":"2024-09-22T03:11:08.278389+0000","last_active":"2024-09-22T03:11:08.276778+0000","last_peered":"2024-09-22T03:11:08.276778+0000","last_clean":"2024-09-22T03:11:08.276778+0000","last_became_active":"2024-09-22T03:11:06.309361+0000","last_became_peered":"2024-09-22T03:11:06.309361+0000","last_unstale":"2024-09-22T03:11:27.408583+0000","last_undegraded":"2024-09-22T03:11:08.276778+0000","last_fullsized":"2024-09-22T03:11:08.276778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:13.388416+0000","last_change":"2024-09-22T03:11:13.388416+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:13.388416+0000","last_undegraded":"2024-09-22T03:11:13.388416+0000","last_fullsized":"2024-09-22T03:11:13.388416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.432900+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:27.432900+0000","last_peered":"2024-09-22T03:11:27.432900+0000","last_clean":"2024-09-22T03:11:27.432900+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:27.432900+0000","last_undegraded":"2024-09-22T03:11:27.432900+0000","last_fullsized":"2024-09-22T03:11:27.432900+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"17'4","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-22T03:11:27.416131+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:11:27.416131+0000","last_peered":"2024-09-22T03:11:27.416131+0000","last_clean":"2024-09-22T03:11:27.416131+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:11:27.416131+0000","last_undegraded":"2024-09-22T03:11:27.416131+0000","last_fullsized":"2024-09-22T03:11:27.416131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:06.266491+0000","last_change":"2024-09-22T03:11:27.434769+0000","last_active":"2024-09-22T03:11:05.253234+0000","last_peered":"2024-09-22T03:11:05.253234+0000","last_clean":"2024-09-22T03:11:05.253234+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:06.266491+0000","last_undegraded":"2024-09-22T03:11:06.266491+0000","last_fullsized":"2024-09-22T03:11:06.266491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-22T03:11:27.454594+0000","last_change":"2024-09-22T03:11:27.454594+0000","last_active":"2024-09-22T03:11:13.388416+0000","last_peered":"2024-09-22T03:11:13.388416+0000","last_clean":"2024-09-22T03:11:13.388416+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:27.454594+0000","last_undegraded":"2024-09-22T03:11:27.454594+0000","last_fullsized":"2024-09-22T03:11:27.454594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-22T03:11:06.266485+0000","last_change":"2024-09-22T03:11:27.423555+0000","last_active":"2024-09-22T03:11:05.253234+0000","last_peered":"2024-09-22T03:11:05.253234+0000","last_clean":"2024-09-22T03:11:05.253234+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:11:06.266485+0000","last_undegraded":"2024-09-22T03:11:06.266485+0000","last_fullsized":"2024-09-22T03:11:06.266485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":19464192,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":19008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":51904512,"data_compressed":2463876,"data_compressed_allocated":3244032,"data_compressed_original":51904512,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":37,"acting":37,"num_store_stats":3},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":20,"acting":20,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27012,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344828,"statfs":{"total":96636764160,"available":96609103872,"internally_reserved":0,"allocated":184320,"data_stored":631079,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27072,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":245760,"data_stored":1221447,"data_compressed":57957,"data_compressed_allocated":77824,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27024,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344816,"statfs":{"total":96636764160,"available":96609091584,"internally_reserved":0,"allocated":184320,"data_stored":621125,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705693,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28952,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94342888,"statfs":{"total":96636764160,"available":96607117312,"internally_reserved":0,"allocated":2170880,"data_stored":31578608,"data_compressed":1498275,"data_compressed_allocated":1974272,"data_compressed_original":31506432,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3170304,"data_stored":50724864,"data_compressed":2407906,"data_compressed_allocated":3170304,"data_compressed_original":50724864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":589824,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":589824,"data_compressed":27985,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:35.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:11:35.477714+0000 min lat: 0.00921296 max lat: 14.2225 avg lat: 0.130401 2024-09-22T03:11:35.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:11:35.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 20 16 2468 2452 7.66167 27.5625 0.0253049 0.130401 2024-09-22T03:11:36.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 21 16 2928 2912 8.66574 28.75 0.0308317 0.115275 2024-09-22T03:11:36.574 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:11:36.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:36.715+0000 7f8ef286c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:36.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:36.724+0000 7f8ef286c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:37.478 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 22 16 3366 3350 9.51605 27.375 0.034001 0.104993 2024-09-22T03:11:37.763 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:11:37.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:37.921+0000 7f1c42023640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:37.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:37.934+0000 7f1c42023640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:38.291 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:11:38.292 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:11:38.313 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-22T03:11:37.228870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":94962227,"num_objects":1444,"num_object_clones":0,"num_object_copies":4332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1579,"num_write_kb":94056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1524,"ondisk_log_size":1524,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":133196,"kb_used_data":13324,"kb_used_omap":0,"kb_used_meta":119808,"kb_avail":377354164,"statfs":{"total":386547056640,"available":386410663936,"internally_reserved":0,"allocated":13643776,"data_stored":205491252,"data_compressed":9749567,"data_compressed_allocated":12840960,"data_compressed_original":205209600,"omap_allocated":0,"internal_metadata":122683392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":93782016,"num_objects":1431,"num_object_clones":0,"num_object_copies":4293,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":91584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.809588"},"pg_stats":[{"pgid":"3.f","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.133267+0000","last_change":"2024-09-22T03:11:29.505342+0000","last_active":"2024-09-22T03:11:32.133267+0000","last_peered":"2024-09-22T03:11:32.133267+0000","last_clean":"2024-09-22T03:11:32.133267+0000","last_became_active":"2024-09-22T03:11:29.496134+0000","last_became_peered":"2024-09-22T03:11:29.496134+0000","last_unstale":"2024-09-22T03:11:32.133267+0000","last_undegraded":"2024-09-22T03:11:32.133267+0000","last_fullsized":"2024-09-22T03:11:32.133267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'86","reported_seq":109,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.144160+0000","last_change":"2024-09-22T03:11:29.513344+0000","last_active":"2024-09-22T03:11:33.144160+0000","last_peered":"2024-09-22T03:11:33.144160+0000","last_clean":"2024-09-22T03:11:33.144160+0000","last_became_active":"2024-09-22T03:11:29.504664+0000","last_became_peered":"2024-09-22T03:11:29.504664+0000","last_unstale":"2024-09-22T03:11:33.144160+0000","last_undegraded":"2024-09-22T03:11:33.144160+0000","last_fullsized":"2024-09-22T03:11:33.144160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'95","reported_seq":114,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.235712+0000","last_change":"2024-09-22T03:11:29.555425+0000","last_active":"2024-09-22T03:11:33.235712+0000","last_peered":"2024-09-22T03:11:33.235712+0000","last_clean":"2024-09-22T03:11:33.235712+0000","last_became_active":"2024-09-22T03:11:29.547257+0000","last_became_peered":"2024-09-22T03:11:29.547257+0000","last_unstale":"2024-09-22T03:11:33.235712+0000","last_undegraded":"2024-09-22T03:11:33.235712+0000","last_fullsized":"2024-09-22T03:11:33.235712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'73","reported_seq":101,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.209856+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:11:32.209856+0000","last_peered":"2024-09-22T03:11:32.209856+0000","last_clean":"2024-09-22T03:11:32.209856+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:11:32.209856+0000","last_undegraded":"2024-09-22T03:11:32.209856+0000","last_fullsized":"2024-09-22T03:11:32.209856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'92","reported_seq":123,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.196295+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:11:33.196295+0000","last_peered":"2024-09-22T03:11:33.196295+0000","last_clean":"2024-09-22T03:11:33.196295+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:11:33.196295+0000","last_undegraded":"2024-09-22T03:11:33.196295+0000","last_fullsized":"2024-09-22T03:11:33.196295+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"22'67","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.132148+0000","last_change":"2024-09-22T03:11:29.503432+0000","last_active":"2024-09-22T03:11:32.132148+0000","last_peered":"2024-09-22T03:11:32.132148+0000","last_clean":"2024-09-22T03:11:32.132148+0000","last_became_active":"2024-09-22T03:11:29.492577+0000","last_became_peered":"2024-09-22T03:11:29.492577+0000","last_unstale":"2024-09-22T03:11:32.132148+0000","last_undegraded":"2024-09-22T03:11:32.132148+0000","last_fullsized":"2024-09-22T03:11:32.132148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"22'78","reported_seq":98,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.233358+0000","last_change":"2024-09-22T03:11:29.523421+0000","last_active":"2024-09-22T03:11:32.233358+0000","last_peered":"2024-09-22T03:11:32.233358+0000","last_clean":"2024-09-22T03:11:32.233358+0000","last_became_active":"2024-09-22T03:11:29.515869+0000","last_became_peered":"2024-09-22T03:11:29.515869+0000","last_unstale":"2024-09-22T03:11:32.233358+0000","last_undegraded":"2024-09-22T03:11:32.233358+0000","last_fullsized":"2024-09-22T03:11:32.233358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'87","reported_seq":106,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.236916+0000","last_change":"2024-09-22T03:11:29.547409+0000","last_active":"2024-09-22T03:11:33.236916+0000","last_peered":"2024-09-22T03:11:33.236916+0000","last_clean":"2024-09-22T03:11:33.236916+0000","last_became_active":"2024-09-22T03:11:29.540128+0000","last_became_peered":"2024-09-22T03:11:29.540128+0000","last_unstale":"2024-09-22T03:11:33.236916+0000","last_undegraded":"2024-09-22T03:11:33.236916+0000","last_fullsized":"2024-09-22T03:11:33.236916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.450727+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:32.450727+0000","last_peered":"2024-09-22T03:11:32.450727+0000","last_clean":"2024-09-22T03:11:32.450727+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:32.450727+0000","last_undegraded":"2024-09-22T03:11:32.450727+0000","last_fullsized":"2024-09-22T03:11:32.450727+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"23'108","reported_seq":127,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.291206+0000","last_change":"2024-09-22T03:11:29.601655+0000","last_active":"2024-09-22T03:11:33.291206+0000","last_peered":"2024-09-22T03:11:33.291206+0000","last_clean":"2024-09-22T03:11:33.291206+0000","last_became_active":"2024-09-22T03:11:29.588642+0000","last_became_peered":"2024-09-22T03:11:29.588642+0000","last_unstale":"2024-09-22T03:11:33.291206+0000","last_undegraded":"2024-09-22T03:11:33.291206+0000","last_fullsized":"2024-09-22T03:11:33.291206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.444471+0000","last_change":"2024-09-22T03:11:30.445307+0000","last_active":"2024-09-22T03:11:31.444471+0000","last_peered":"2024-09-22T03:11:31.444471+0000","last_clean":"2024-09-22T03:11:31.444471+0000","last_became_active":"2024-09-22T03:11:29.457496+0000","last_became_peered":"2024-09-22T03:11:29.457496+0000","last_unstale":"2024-09-22T03:11:31.444471+0000","last_undegraded":"2024-09-22T03:11:31.444471+0000","last_fullsized":"2024-09-22T03:11:31.444471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'85","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.031170+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:11:33.031170+0000","last_peered":"2024-09-22T03:11:33.031170+0000","last_clean":"2024-09-22T03:11:33.031170+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:11:33.031170+0000","last_undegraded":"2024-09-22T03:11:33.031170+0000","last_fullsized":"2024-09-22T03:11:33.031170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.431587+0000","last_change":"2024-09-22T03:11:30.441560+0000","last_active":"2024-09-22T03:11:32.431587+0000","last_peered":"2024-09-22T03:11:32.431587+0000","last_clean":"2024-09-22T03:11:32.431587+0000","last_became_active":"2024-09-22T03:11:29.480249+0000","last_became_peered":"2024-09-22T03:11:29.480249+0000","last_unstale":"2024-09-22T03:11:32.431587+0000","last_undegraded":"2024-09-22T03:11:32.431587+0000","last_fullsized":"2024-09-22T03:11:32.431587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'102","reported_seq":125,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.245756+0000","last_change":"2024-09-22T03:11:29.487178+0000","last_active":"2024-09-22T03:11:33.245756+0000","last_peered":"2024-09-22T03:11:33.245756+0000","last_clean":"2024-09-22T03:11:33.245756+0000","last_became_active":"2024-09-22T03:11:29.480276+0000","last_became_peered":"2024-09-22T03:11:29.480276+0000","last_unstale":"2024-09-22T03:11:33.245756+0000","last_undegraded":"2024-09-22T03:11:33.245756+0000","last_fullsized":"2024-09-22T03:11:33.245756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.455212+0000","last_change":"2024-09-22T03:11:30.461677+0000","last_active":"2024-09-22T03:11:32.455212+0000","last_peered":"2024-09-22T03:11:32.455212+0000","last_clean":"2024-09-22T03:11:32.455212+0000","last_became_active":"2024-09-22T03:11:29.490989+0000","last_became_peered":"2024-09-22T03:11:29.490989+0000","last_unstale":"2024-09-22T03:11:32.455212+0000","last_undegraded":"2024-09-22T03:11:32.455212+0000","last_fullsized":"2024-09-22T03:11:32.455212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-22T03:11:31.448251+0000","last_change":"2024-09-22T03:11:29.502756+0000","last_active":"2024-09-22T03:11:31.448251+0000","last_peered":"2024-09-22T03:11:31.448251+0000","last_clean":"2024-09-22T03:11:31.448251+0000","last_became_active":"2024-09-22T03:11:29.498892+0000","last_became_peered":"2024-09-22T03:11:29.498892+0000","last_unstale":"2024-09-22T03:11:31.448251+0000","last_undegraded":"2024-09-22T03:11:31.448251+0000","last_fullsized":"2024-09-22T03:11:31.448251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'103","reported_seq":126,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.239309+0000","last_change":"2024-09-22T03:11:29.547647+0000","last_active":"2024-09-22T03:11:33.239309+0000","last_peered":"2024-09-22T03:11:33.239309+0000","last_clean":"2024-09-22T03:11:33.239309+0000","last_became_active":"2024-09-22T03:11:29.538561+0000","last_became_peered":"2024-09-22T03:11:29.538561+0000","last_unstale":"2024-09-22T03:11:33.239309+0000","last_undegraded":"2024-09-22T03:11:33.239309+0000","last_fullsized":"2024-09-22T03:11:33.239309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'89","reported_seq":112,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.208575+0000","last_change":"2024-09-22T03:11:29.498850+0000","last_active":"2024-09-22T03:11:33.208575+0000","last_peered":"2024-09-22T03:11:33.208575+0000","last_clean":"2024-09-22T03:11:33.208575+0000","last_became_active":"2024-09-22T03:11:29.491879+0000","last_became_peered":"2024-09-22T03:11:29.491879+0000","last_unstale":"2024-09-22T03:11:33.208575+0000","last_undegraded":"2024-09-22T03:11:33.208575+0000","last_fullsized":"2024-09-22T03:11:33.208575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.463864+0000","last_change":"2024-09-22T03:11:30.462372+0000","last_active":"2024-09-22T03:11:32.463864+0000","last_peered":"2024-09-22T03:11:32.463864+0000","last_clean":"2024-09-22T03:11:32.463864+0000","last_became_active":"2024-09-22T03:11:29.527808+0000","last_became_peered":"2024-09-22T03:11:29.527808+0000","last_unstale":"2024-09-22T03:11:32.463864+0000","last_undegraded":"2024-09-22T03:11:32.463864+0000","last_fullsized":"2024-09-22T03:11:32.463864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'101","reported_seq":124,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.114975+0000","last_change":"2024-09-22T03:11:29.515961+0000","last_active":"2024-09-22T03:11:33.114975+0000","last_peered":"2024-09-22T03:11:33.114975+0000","last_clean":"2024-09-22T03:11:33.114975+0000","last_became_active":"2024-09-22T03:11:29.496174+0000","last_became_peered":"2024-09-22T03:11:29.496174+0000","last_unstale":"2024-09-22T03:11:33.114975+0000","last_undegraded":"2024-09-22T03:11:33.114975+0000","last_fullsized":"2024-09-22T03:11:33.114975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.434431+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:32.434431+0000","last_peered":"2024-09-22T03:11:32.434431+0000","last_clean":"2024-09-22T03:11:32.434431+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:32.434431+0000","last_undegraded":"2024-09-22T03:11:32.434431+0000","last_fullsized":"2024-09-22T03:11:32.434431+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'91","reported_seq":122,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.214403+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:11:33.214403+0000","last_peered":"2024-09-22T03:11:33.214403+0000","last_clean":"2024-09-22T03:11:33.214403+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:11:33.214403+0000","last_undegraded":"2024-09-22T03:11:33.214403+0000","last_fullsized":"2024-09-22T03:11:33.214403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.464267+0000","last_change":"2024-09-22T03:11:30.461276+0000","last_active":"2024-09-22T03:11:32.464267+0000","last_peered":"2024-09-22T03:11:32.464267+0000","last_clean":"2024-09-22T03:11:32.464267+0000","last_became_active":"2024-09-22T03:11:29.551295+0000","last_became_peered":"2024-09-22T03:11:29.551295+0000","last_unstale":"2024-09-22T03:11:32.464267+0000","last_undegraded":"2024-09-22T03:11:32.464267+0000","last_fullsized":"2024-09-22T03:11:32.464267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'100","reported_seq":119,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:33.197524+0000","last_change":"2024-09-22T03:11:29.535585+0000","last_active":"2024-09-22T03:11:33.197524+0000","last_peered":"2024-09-22T03:11:33.197524+0000","last_clean":"2024-09-22T03:11:33.197524+0000","last_became_active":"2024-09-22T03:11:29.524209+0000","last_became_peered":"2024-09-22T03:11:29.524209+0000","last_unstale":"2024-09-22T03:11:33.197524+0000","last_undegraded":"2024-09-22T03:11:33.197524+0000","last_fullsized":"2024-09-22T03:11:33.197524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.460648+0000","last_change":"2024-09-22T03:11:30.460143+0000","last_active":"2024-09-22T03:11:32.460648+0000","last_peered":"2024-09-22T03:11:32.460648+0000","last_clean":"2024-09-22T03:11:32.460648+0000","last_became_active":"2024-09-22T03:11:29.508700+0000","last_became_peered":"2024-09-22T03:11:29.508700+0000","last_unstale":"2024-09-22T03:11:32.460648+0000","last_undegraded":"2024-09-22T03:11:32.460648+0000","last_fullsized":"2024-09-22T03:11:32.460648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":94371840,"num_objects":1440,"num_object_clones":0,"num_object_copies":4320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1440,"num_write_kb":92160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17760256,"data_stored":284164096,"data_compressed":13490302,"data_compressed_allocated":17760256,"data_compressed_original":284164096,"omap_allocated":0,"internal_metadata":0},"log_size":1424,"ondisk_log_size":1424,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35052,"kb_used_data":4060,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94336788,"statfs":{"total":96636764160,"available":96600870912,"internally_reserved":0,"allocated":4157440,"data_stored":63562924,"data_compressed":3014567,"data_compressed_allocated":3969024,"data_compressed_original":63504384,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34240,"kb_used_data":3248,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94337600,"statfs":{"total":96636764160,"available":96601702400,"internally_reserved":0,"allocated":3325952,"data_stored":49866444,"data_compressed":2366348,"data_compressed_allocated":3117056,"data_compressed_original":49790976,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":19,"seq":81604378628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34952,"kb_used_data":3896,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94336888,"statfs":{"total":96636764160,"available":96600973312,"internally_reserved":0,"allocated":3989504,"data_stored":60483276,"data_compressed":2870377,"data_compressed_allocated":3780608,"data_compressed_original":60407808,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705693,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28952,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94342888,"statfs":{"total":96636764160,"available":96607117312,"internally_reserved":0,"allocated":2170880,"data_stored":31578608,"data_compressed":1498275,"data_compressed_allocated":1974272,"data_compressed_original":31506432,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3170304,"data_stored":50724864,"data_compressed":2407906,"data_compressed_allocated":3170304,"data_compressed_original":50724864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4243456,"data_stored":67895296,"data_compressed":3223296,"data_compressed_allocated":4243456,"data_compressed_original":67895296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5468160,"data_stored":87490560,"data_compressed":4153559,"data_compressed_allocated":5468160,"data_compressed_original":87490560,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:11:38.314 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:11:38.314 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:11:38.315 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 3, 1] 2024-09-22T03:11:38.315 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:11:38.315 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:11:38.400 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:11:38.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 23 16 3716 3700 10.0533 21.875 0.01064 0.0986748 2024-09-22T03:11:38.927 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:11:39.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:39.028+0000 7f4232997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:39.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:39.033+0000 7f4232997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:39.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 24 16 3716 3700 9.63438 0 - 0.0986748 2024-09-22T03:11:40.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 25 16 3716 3700 9.24901 0 - 0.0986748 2024-09-22T03:11:41.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 26 16 3716 3700 8.89326 0 - 0.0986748 2024-09-22T03:11:42.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 27 16 3716 3700 8.56387 0 - 0.0986748 2024-09-22T03:11:43.401 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-22T03:11:43.401 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:11:43.401 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:11:43.402 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:11:43.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 28 16 3716 3700 8.25801 0 - 0.0986748 2024-09-22T03:11:43.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:43.514+0000 7fc49aeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:43.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:43.522+0000 7fc49aeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:11:43.536 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:11:43.537 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:11:43.537 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:11:43.539 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:11:43.540 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:11:43.540 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:11:43.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:43.640+0000 7fb05df97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:43.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:11:43.648+0000 7fb05df97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:11:44.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 29 16 3716 3700 7.97324 0 - 0.0986748 2024-09-22T03:11:45.479 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 30 16 3716 3700 7.70746 0 - 0.0986748 2024-09-22T03:11:46.480 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 31 16 3716 3700 7.45882 0 - 0.0986748 2024-09-22T03:11:47.480 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 32 16 3716 3700 7.22572 0 - 0.0986748 2024-09-22T03:11:48.480 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 33 16 3716 3700 7.00676 0 - 0.0986748 2024-09-22T03:11:49.480 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 34 16 3716 3700 6.80068 0 - 0.0986748 2024-09-22T03:11:50.480 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 35 16 3716 3700 6.60637 0 - 0.0986748 2024-09-22T03:11:51.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 36 16 3716 3700 6.42286 0 - 0.0986748 2024-09-22T03:11:52.480 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 37 16 3716 3700 6.24926 0 - 0.0986748 2024-09-22T03:11:53.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 38 16 3716 3700 6.08481 0 - 0.0986748 2024-09-22T03:11:54.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 39 16 3716 3700 5.92878 0 - 0.0986748 2024-09-22T03:11:55.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:11:55.480335+0000 min lat: 0.00921296 max lat: 14.2225 avg lat: 0.0986748 2024-09-22T03:11:55.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:11:55.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 40 16 3716 3700 5.78056 0 - 0.0986748 2024-09-22T03:11:56.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 41 16 3716 3700 5.63956 0 - 0.0986748 2024-09-22T03:11:57.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 42 16 3716 3700 5.50528 0 - 0.0986748 2024-09-22T03:11:58.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 43 16 3716 3700 5.37725 0 - 0.0986748 2024-09-22T03:11:59.481 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 44 16 3716 3700 5.25504 0 - 0.0986748 2024-09-22T03:12:00.482 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 45 16 3716 3700 5.13826 0 - 0.0986748 2024-09-22T03:12:01.482 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 46 16 3716 3700 5.02656 0 - 0.0986748 2024-09-22T03:12:02.482 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 47 16 3716 3700 4.91961 0 - 0.0986748 2024-09-22T03:12:03.482 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 48 16 3716 3700 4.81711 0 - 0.0986748 2024-09-22T03:12:04.482 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 49 16 3716 3700 4.7188 0 - 0.0986748 2024-09-22T03:12:05.482 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 50 16 3716 3700 4.62442 0 - 0.0986748 2024-09-22T03:12:06.482 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 51 16 3716 3700 4.53375 0 - 0.0986748 2024-09-22T03:12:07.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 52 16 3716 3700 4.44656 0 - 0.0986748 2024-09-22T03:12:08.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 53 16 3716 3700 4.36266 0 - 0.0986748 2024-09-22T03:12:09.021 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:09.021 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:09.025 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:09.026 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:12:09.026 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:12:09.113 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:09.114 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:09.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:09.145+0000 7f57f0397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:09.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:09.152+0000 7f57f0397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:09.166 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:12:09.166 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:12:09.167 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:12:09.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:12:09.170 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:12:09.170 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:12:09.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:09.308+0000 7fdd1d86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:09.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:09.316+0000 7fdd1d86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:09.330 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:12:09.330 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:12:09.330 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:12:09.330 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:12:09.330 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:12:09.331 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:12:09.331 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:12:09.331 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:12:09.331 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:12:09.331 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:12:09.331 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:12:09.331 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:12:09.333 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:12:09.334 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:12:09.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 54 16 3716 3700 4.28187 0 - 0.0986748 2024-09-22T03:12:10.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 55 16 3867 3851 4.37559 0.294922 0.0416009 0.22848 2024-09-22T03:12:11.114 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:11.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:11.224+0000 7f060feb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:11.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:11.230+0000 7f060feb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:11.257 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:11.339 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:11.340 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:11.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 56 16 4272 4256 4.74941 25.3125 0.0272427 0.21047 2024-09-22T03:12:12.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 57 16 4700 4684 5.13533 26.75 0.0243864 0.194659 2024-09-22T03:12:13.341 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:13.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:13.477+0000 7fd81c46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:13.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 58 16 5131 5115 5.51118 26.9375 0.0258356 0.181379 2024-09-22T03:12:13.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:13.482+0000 7fd81c46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:13.507 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:13.598 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:13.598 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:14.334 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-22T03:12:14.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:12:14.334 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:14.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:14.467+0000 7f463ea6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:14.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:14.474+0000 7f463ea6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:14.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 59 16 5538 5522 5.84886 25.4375 0.0624578 0.170896 2024-09-22T03:12:14.693 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:14.694 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:14.709 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-09-22T03:12:13.236390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235471411,"num_objects":3588,"num_object_clones":0,"num_object_copies":10764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3723,"num_write_kb":231272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3668,"ondisk_log_size":3668,"up":68,"acting":68,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":179904,"kb_used_data":39232,"kb_used_omap":0,"kb_used_meta":140608,"kb_avail":377307456,"statfs":{"total":386547056640,"available":386362834944,"internally_reserved":0,"allocated":40173568,"data_stored":629201545,"data_compressed":29868426,"data_compressed_allocated":39321600,"data_compressed_original":628899840,"omap_allocated":0,"internal_metadata":143982592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28770304,"num_objects":439,"num_object_clones":0,"num_object_copies":1317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":439,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001414"},"pg_stats":[{"pgid":"3.f","version":"25'288","reported_seq":318,"reported_epoch":25,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:12:12.208482+0000","last_change":"2024-09-22T03:12:10.096291+0000","last_active":"2024-09-22T03:12:12.208482+0000","last_peered":"2024-09-22T03:12:12.208482+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:10.096291+0000","last_became_peered":"2024-09-22T03:12:10.096291+0000","last_unstale":"2024-09-22T03:12:12.208482+0000","last_undegraded":"2024-09-22T03:12:10.036594+0000","last_fullsized":"2024-09-22T03:12:10.035904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":289,"num_object_clones":0,"num_object_copies":867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":289,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":289}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'218","reported_seq":241,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.237827+0000","last_change":"2024-09-22T03:11:29.513344+0000","last_active":"2024-09-22T03:11:38.237827+0000","last_peered":"2024-09-22T03:11:38.237827+0000","last_clean":"2024-09-22T03:11:38.237827+0000","last_became_active":"2024-09-22T03:11:29.504664+0000","last_became_peered":"2024-09-22T03:11:29.504664+0000","last_unstale":"2024-09-22T03:11:38.237827+0000","last_undegraded":"2024-09-22T03:11:38.237827+0000","last_fullsized":"2024-09-22T03:11:38.237827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":219,"num_object_clones":0,"num_object_copies":657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'95","reported_seq":114,"reported_epoch":23,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:33.235712+0000","last_change":"2024-09-22T03:11:29.555425+0000","last_active":"2024-09-22T03:11:33.235712+0000","last_peered":"2024-09-22T03:11:33.235712+0000","last_clean":"2024-09-22T03:11:33.235712+0000","last_became_active":"2024-09-22T03:11:29.547257+0000","last_became_peered":"2024-09-22T03:11:29.547257+0000","last_unstale":"2024-09-22T03:12:09.017908+0000","last_undegraded":"2024-09-22T03:11:33.235712+0000","last_fullsized":"2024-09-22T03:11:33.235712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"25'276","reported_seq":313,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-22T03:12:12.144963+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:12.144963+0000","last_peered":"2024-09-22T03:12:12.144963+0000","last_clean":"2024-09-22T03:12:12.144963+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:12.144963+0000","last_undegraded":"2024-09-22T03:12:12.144963+0000","last_fullsized":"2024-09-22T03:12:12.144963+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":277,"num_object_clones":0,"num_object_copies":831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":277,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'246","reported_seq":277,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.294025+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:11:38.294025+0000","last_peered":"2024-09-22T03:11:38.294025+0000","last_clean":"2024-09-22T03:11:38.294025+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:11:38.294025+0000","last_undegraded":"2024-09-22T03:11:38.294025+0000","last_fullsized":"2024-09-22T03:11:38.294025+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16187392,"num_objects":247,"num_object_clones":0,"num_object_copies":741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":247,"num_write_kb":15808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'265","reported_seq":295,"reported_epoch":25,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:12:12.158478+0000","last_change":"2024-09-22T03:12:10.069384+0000","last_active":"2024-09-22T03:12:12.158478+0000","last_peered":"2024-09-22T03:12:12.158478+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:10.069384+0000","last_became_peered":"2024-09-22T03:12:10.069384+0000","last_unstale":"2024-09-22T03:12:12.158478+0000","last_undegraded":"2024-09-22T03:12:10.034876+0000","last_fullsized":"2024-09-22T03:12:10.034232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17432576,"num_objects":266,"num_object_clones":0,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":266,"num_write_kb":17024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":266}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'322","reported_seq":352,"reported_epoch":25,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:12:12.226665+0000","last_change":"2024-09-22T03:12:10.065362+0000","last_active":"2024-09-22T03:12:12.226665+0000","last_peered":"2024-09-22T03:12:12.226665+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:10.065362+0000","last_became_peered":"2024-09-22T03:12:10.065362+0000","last_unstale":"2024-09-22T03:12:12.226665+0000","last_undegraded":"2024-09-22T03:12:10.036032+0000","last_fullsized":"2024-09-22T03:12:10.035402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":323,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":323}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'87","reported_seq":106,"reported_epoch":23,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:33.236916+0000","last_change":"2024-09-22T03:11:29.547409+0000","last_active":"2024-09-22T03:11:33.236916+0000","last_peered":"2024-09-22T03:11:33.236916+0000","last_clean":"2024-09-22T03:11:33.236916+0000","last_became_active":"2024-09-22T03:11:29.540128+0000","last_became_peered":"2024-09-22T03:11:29.540128+0000","last_unstale":"2024-09-22T03:12:09.017911+0000","last_undegraded":"2024-09-22T03:11:33.236916+0000","last_fullsized":"2024-09-22T03:11:33.236916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.450727+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:11:32.450727+0000","last_peered":"2024-09-22T03:11:32.450727+0000","last_clean":"2024-09-22T03:11:32.450727+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:11:32.450727+0000","last_undegraded":"2024-09-22T03:11:32.450727+0000","last_fullsized":"2024-09-22T03:11:32.450727+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'316","reported_seq":342,"reported_epoch":25,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:12:12.235937+0000","last_change":"2024-09-22T03:12:10.082380+0000","last_active":"2024-09-22T03:12:12.235937+0000","last_peered":"2024-09-22T03:12:12.235937+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:10.082380+0000","last_became_peered":"2024-09-22T03:12:10.082380+0000","last_unstale":"2024-09-22T03:12:12.235937+0000","last_undegraded":"2024-09-22T03:12:10.033948+0000","last_fullsized":"2024-09-22T03:12:10.033385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":317,"num_object_clones":0,"num_object_copies":951,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":317,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.059199+0000","last_change":"2024-09-22T03:12:10.058777+0000","last_active":"2024-09-22T03:12:10.059199+0000","last_peered":"2024-09-22T03:12:10.059199+0000","last_clean":"2024-09-22T03:11:32.452176+0000","last_became_active":"2024-09-22T03:12:10.058777+0000","last_became_peered":"2024-09-22T03:12:10.058777+0000","last_unstale":"2024-09-22T03:12:10.059199+0000","last_undegraded":"2024-09-22T03:12:10.059199+0000","last_fullsized":"2024-09-22T03:12:10.029219+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'222","reported_seq":253,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.232261+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:11:38.232261+0000","last_peered":"2024-09-22T03:11:38.232261+0000","last_clean":"2024-09-22T03:11:38.232261+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:11:38.232261+0000","last_undegraded":"2024-09-22T03:11:38.232261+0000","last_fullsized":"2024-09-22T03:11:38.232261+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14614528,"num_objects":223,"num_object_clones":0,"num_object_copies":669,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":14272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.431587+0000","last_change":"2024-09-22T03:11:30.441560+0000","last_active":"2024-09-22T03:11:32.431587+0000","last_peered":"2024-09-22T03:11:32.431587+0000","last_clean":"2024-09-22T03:11:32.431587+0000","last_became_active":"2024-09-22T03:11:29.480249+0000","last_became_peered":"2024-09-22T03:11:29.480249+0000","last_unstale":"2024-09-22T03:11:32.431587+0000","last_undegraded":"2024-09-22T03:11:32.431587+0000","last_fullsized":"2024-09-22T03:11:32.431587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'234","reported_seq":257,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.251708+0000","last_change":"2024-09-22T03:11:29.487178+0000","last_active":"2024-09-22T03:11:38.251708+0000","last_peered":"2024-09-22T03:11:38.251708+0000","last_clean":"2024-09-22T03:11:38.251708+0000","last_became_active":"2024-09-22T03:11:29.480276+0000","last_became_peered":"2024-09-22T03:11:29.480276+0000","last_unstale":"2024-09-22T03:11:38.251708+0000","last_undegraded":"2024-09-22T03:11:38.251708+0000","last_fullsized":"2024-09-22T03:11:38.251708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15400960,"num_objects":235,"num_object_clones":0,"num_object_copies":705,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":235,"num_write_kb":15040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.455212+0000","last_change":"2024-09-22T03:11:30.461677+0000","last_active":"2024-09-22T03:11:32.455212+0000","last_peered":"2024-09-22T03:11:32.455212+0000","last_clean":"2024-09-22T03:11:32.455212+0000","last_became_active":"2024-09-22T03:11:29.490989+0000","last_became_peered":"2024-09-22T03:11:29.490989+0000","last_unstale":"2024-09-22T03:11:32.455212+0000","last_undegraded":"2024-09-22T03:11:32.455212+0000","last_fullsized":"2024-09-22T03:11:32.455212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":177,"reported_epoch":25,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:12:10.061250+0000","last_change":"2024-09-22T03:12:10.060853+0000","last_active":"2024-09-22T03:12:10.061250+0000","last_peered":"2024-09-22T03:12:10.061250+0000","last_clean":"2024-09-22T03:11:32.455841+0000","last_became_active":"2024-09-22T03:12:10.060853+0000","last_became_peered":"2024-09-22T03:12:10.060853+0000","last_unstale":"2024-09-22T03:12:10.061250+0000","last_undegraded":"2024-09-22T03:12:10.032727+0000","last_fullsized":"2024-09-22T03:12:10.032127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'225","reported_seq":248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.261640+0000","last_change":"2024-09-22T03:11:29.547647+0000","last_active":"2024-09-22T03:11:38.261640+0000","last_peered":"2024-09-22T03:11:38.261640+0000","last_clean":"2024-09-22T03:11:38.261640+0000","last_became_active":"2024-09-22T03:11:29.538561+0000","last_became_peered":"2024-09-22T03:11:29.538561+0000","last_unstale":"2024-09-22T03:11:38.261640+0000","last_undegraded":"2024-09-22T03:11:38.261640+0000","last_fullsized":"2024-09-22T03:11:38.261640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14811136,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":226,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'220","reported_seq":243,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.151210+0000","last_change":"2024-09-22T03:11:29.498850+0000","last_active":"2024-09-22T03:11:38.151210+0000","last_peered":"2024-09-22T03:11:38.151210+0000","last_clean":"2024-09-22T03:11:38.151210+0000","last_became_active":"2024-09-22T03:11:29.491879+0000","last_became_peered":"2024-09-22T03:11:29.491879+0000","last_unstale":"2024-09-22T03:11:38.151210+0000","last_undegraded":"2024-09-22T03:11:38.151210+0000","last_fullsized":"2024-09-22T03:11:38.151210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":221,"num_write_kb":14144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.074574+0000","last_change":"2024-09-22T03:12:10.074230+0000","last_active":"2024-09-22T03:12:10.074574+0000","last_peered":"2024-09-22T03:12:10.074574+0000","last_clean":"2024-09-22T03:11:06.315471+0000","last_became_active":"2024-09-22T03:12:10.074230+0000","last_became_peered":"2024-09-22T03:12:10.074230+0000","last_unstale":"2024-09-22T03:12:10.074574+0000","last_undegraded":"2024-09-22T03:12:10.074574+0000","last_fullsized":"2024-09-22T03:12:10.032035+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'228","reported_seq":251,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.255430+0000","last_change":"2024-09-22T03:11:29.515961+0000","last_active":"2024-09-22T03:11:38.255430+0000","last_peered":"2024-09-22T03:11:38.255430+0000","last_clean":"2024-09-22T03:11:38.255430+0000","last_became_active":"2024-09-22T03:11:29.496174+0000","last_became_peered":"2024-09-22T03:11:29.496174+0000","last_unstale":"2024-09-22T03:11:38.255430+0000","last_undegraded":"2024-09-22T03:11:38.255430+0000","last_fullsized":"2024-09-22T03:11:38.255430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":229,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:32.434431+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:11:32.434431+0000","last_peered":"2024-09-22T03:11:32.434431+0000","last_clean":"2024-09-22T03:11:32.434431+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:11:32.434431+0000","last_undegraded":"2024-09-22T03:11:32.434431+0000","last_fullsized":"2024-09-22T03:11:32.434431+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'226","reported_seq":257,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:11:38.320589+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:11:38.320589+0000","last_peered":"2024-09-22T03:11:38.320589+0000","last_clean":"2024-09-22T03:11:38.320589+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:11:38.320589+0000","last_undegraded":"2024-09-22T03:11:38.320589+0000","last_fullsized":"2024-09-22T03:11:38.320589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14876672,"num_objects":227,"num_object_clones":0,"num_object_copies":681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":14528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":23,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:32.464267+0000","last_change":"2024-09-22T03:11:30.461276+0000","last_active":"2024-09-22T03:11:32.464267+0000","last_peered":"2024-09-22T03:11:32.464267+0000","last_clean":"2024-09-22T03:11:32.464267+0000","last_became_active":"2024-09-22T03:11:29.551295+0000","last_became_peered":"2024-09-22T03:11:29.551295+0000","last_unstale":"2024-09-22T03:12:09.017941+0000","last_undegraded":"2024-09-22T03:11:32.464267+0000","last_fullsized":"2024-09-22T03:11:32.464267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'100","reported_seq":119,"reported_epoch":23,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:33.197524+0000","last_change":"2024-09-22T03:11:29.535585+0000","last_active":"2024-09-22T03:11:33.197524+0000","last_peered":"2024-09-22T03:11:33.197524+0000","last_clean":"2024-09-22T03:11:33.197524+0000","last_became_active":"2024-09-22T03:11:29.524209+0000","last_became_peered":"2024-09-22T03:11:29.524209+0000","last_unstale":"2024-09-22T03:12:09.017944+0000","last_undegraded":"2024-09-22T03:11:33.197524+0000","last_fullsized":"2024-09-22T03:11:33.197524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":23,"state":"stale+active+clean","last_fresh":"2024-09-22T03:11:32.460648+0000","last_change":"2024-09-22T03:11:30.460143+0000","last_active":"2024-09-22T03:11:32.460648+0000","last_peered":"2024-09-22T03:11:32.460648+0000","last_clean":"2024-09-22T03:11:32.460648+0000","last_became_active":"2024-09-22T03:11:29.508700+0000","last_became_peered":"2024-09-22T03:11:29.508700+0000","last_unstale":"2024-09-22T03:12:09.017947+0000","last_undegraded":"2024-09-22T03:11:32.460648+0000","last_fullsized":"2024-09-22T03:11:32.460648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":234881024,"num_objects":3584,"num_object_clones":0,"num_object_copies":10752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3584,"num_write_kb":229376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41574400,"data_stored":665190400,"data_compressed":31583470,"data_compressed_allocated":41574400,"data_compressed_original":665190400,"omap_allocated":0,"internal_metadata":0},"log_size":3568,"ondisk_log_size":3568,"up":44,"acting":44,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":22,"acting":22,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52456,"kb_used_data":13144,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94319384,"statfs":{"total":96636764160,"available":96583049216,"internally_reserved":0,"allocated":13459456,"data_stored":212267381,"data_compressed":10075695,"data_compressed_allocated":13262848,"data_compressed_original":212205568,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45520,"kb_used_data":10368,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94326320,"statfs":{"total":96636764160,"available":96590151680,"internally_reserved":0,"allocated":10616832,"data_stored":166392725,"data_compressed":7899484,"data_compressed_allocated":10399744,"data_compressed_original":166313984,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":19,"seq":81604378628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34952,"kb_used_data":3896,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94336888,"statfs":{"total":96636764160,"available":96600973312,"internally_reserved":0,"allocated":3989504,"data_stored":60483276,"data_compressed":2870377,"data_compressed_allocated":3780608,"data_compressed_original":60407808,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705733,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46976,"kb_used_data":11824,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94324864,"statfs":{"total":96636764160,"available":96588660736,"internally_reserved":0,"allocated":12107776,"data_stored":190058163,"data_compressed":9022870,"data_compressed_allocated":11878400,"data_compressed_original":189972480,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13074432,"data_stored":209190912,"data_compressed":9932722,"data_compressed_allocated":13074432,"data_compressed_original":209190912,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10358784,"data_stored":165740544,"data_compressed":7869512,"data_compressed_allocated":10358784,"data_compressed_original":165740544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13262848,"data_stored":212205568,"data_compressed":10075695,"data_compressed_allocated":13262848,"data_compressed_original":212205568,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:15.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:12:15.482850+0000 min lat: 0.00921296 max lat: 31.8821 avg lat: 0.162085 2024-09-22T03:12:15.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:12:15.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 60 16 5937 5921 6.16696 24.9375 0.026264 0.162085 2024-09-22T03:12:15.599 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:15.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:15.709+0000 7f9f2a458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:15.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:15.716+0000 7f9f2a458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:15.745 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:15.815 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:15.816 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:16.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 61 16 6358 6342 6.49716 26.3125 0.0461583 0.153876 2024-09-22T03:12:17.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 62 16 6787 6771 6.82478 26.8125 0.0416166 0.146489 2024-09-22T03:12:17.817 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:17.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:17.953+0000 7f0d80757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:17.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:17.958+0000 7f0d80757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:17.984 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:18.084 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:18.084 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:18.483 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 63 16 7187 7171 7.11324 25 0.0487646 0.140509 2024-09-22T03:12:19.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 64 16 7590 7574 7.3956 25.1875 0.0533978 0.135164 2024-09-22T03:12:19.711 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-22T03:12:19.711 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:12:19.711 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:12:19.711 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:12:19.714 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:12:19.714 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:12:19.841 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:12:19.845 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:12:19.845 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:12:20.085 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:20.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:20.228+0000 7f40514d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:20.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:20.234+0000 7f40514d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:20.259 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:20.336 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:20.337 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:20.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 65 16 7993 7977 7.66929 25.1875 0.0506066 0.130348 2024-09-22T03:12:21.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 66 16 8370 8354 7.91005 23.5625 0.0325062 0.126377 2024-09-22T03:12:22.337 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:22.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:22.473+0000 7f8ed1e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:22.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:22.481+0000 7f8ed1e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:22.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 67 16 8766 8750 8.16136 24.75 0.0497489 0.12249 2024-09-22T03:12:22.512 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:22.582 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:22.582 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:23.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 68 16 9159 9143 8.4025 24.5625 0.0292988 0.118977 2024-09-22T03:12:24.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 69 16 9546 9530 8.63123 24.1875 0.0324519 0.115819 2024-09-22T03:12:24.583 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:24.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:24.722+0000 7f3744517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:24.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:24.727+0000 7f3744517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:24.752 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:24.833 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:24.833 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:24.846 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:12:24.973 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:12:24.976 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:12:24.976 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:12:25.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 70 16 9921 9905 8.84272 23.4375 0.0354261 0.113048 2024-09-22T03:12:26.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 71 16 10315 10299 9.06497 24.625 0.0298696 0.11026 2024-09-22T03:12:26.835 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:26.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:26.971+0000 7f0b2ff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:26.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:26.977+0000 7f0b2ff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:27.006 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:27.125 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:27.125 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:27.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 72 16 10688 10672 9.26282 23.3125 0.034036 0.107929 2024-09-22T03:12:28.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 73 16 11091 11075 9.48093 25.1875 0.0464559 0.105428 2024-09-22T03:12:29.126 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:29.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:29.259+0000 7f006a6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:29.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:29.268+0000 7f006a6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:29.305 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:12:29.384 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:12:29.384 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:12:29.484 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 74 16 11464 11448 9.66781 23.3125 0.0813924 0.103396 2024-09-22T03:12:29.977 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:12:30.141 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 36, 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.092777+0000" 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.142 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.092808+0000" 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.092952+0000" 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.143 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.093064+0000" 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.144 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.093165+0000" 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.093437+0000" 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.146 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.093605+0000" 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.147 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.093694+0000", 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PGActivationBlocker", 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.5" 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.148 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.149 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.151 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.152 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.153 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 38, 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.154 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096676+0000" 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096681+0000" 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.155 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096863+0000" 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.156 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096954+0000" 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097034+0000" 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.157 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.158 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097241+0000" 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097445+0000", 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 36 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.163 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.165 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 39, 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097707+0000" 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097711+0000" 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097795+0000" 2024-09-22T03:12:30.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097839+0000" 2024-09-22T03:12:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.097897+0000" 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098098+0000", 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "ClientRequest::PGPipeline::await_map", 2024-09-22T03:12:30.170 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 38 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.171 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.172 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.173 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.174 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.175 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.176 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.177 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 43, 2024-09-22T03:12:30.178 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.100702+0000" 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.100707+0000" 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.179 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.101583+0000" 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.180 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.102299+0000" 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.102665+0000", 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PerShardPipeline::create_or_wait_pg", 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 39 2024-09-22T03:12:30.181 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.182 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.183 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.184 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.185 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.186 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.187 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.188 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.189 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.190 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.191 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 46, 2024-09-22T03:12:30.191 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.191 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.191 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.191 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.105918+0000" 2024-09-22T03:12:30.191 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.105926+0000" 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.106085+0000" 2024-09-22T03:12:30.192 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.193 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.106169+0000", 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "ConnectionPipeline::get_pg_mapping", 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 43 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.195 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.196 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.197 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.199 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.200 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.201 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.202 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.203 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.204 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.205 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16777251, 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "0", 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.9", 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.206 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MQuery 3.9 from 0 query_epoch 26 query: query(info 0'0 epoch_sent 26)", 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.116664+0000" 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.116668+0000" 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.207 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.116800+0000" 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.208 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.116912+0000" 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.119077+0000" 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.210 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.120237+0000" 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.211 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.212 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.122716+0000" 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.213 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554467, 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2", 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.5", 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.214 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MNotifyRec 3.5 from 2 notify: (query:26 sent:26 3.5( v 25'732 (0'0,25'732] local-lis/les=24/25 n=732 ec=15/15 lis/c=24/19 les/c/f=25/20/0 sis=26 pruub=12.080904961s) ([19,25] all_participants={1, 2, 3} intervals=[([24,25] acting={2, 3})])) features: 0x3f03cffffffdffff +create_info", 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.119836+0000" 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.119841+0000" 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.215 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.120049+0000" 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.216 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.120174+0000" 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.217 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.124646+0000" 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.218 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.126264+0000" 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.219 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.126665+0000" 2024-09-22T03:12:30.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.221 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554470, 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2", 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.5", 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MNotifyRec 3.5 from 2 notify: (query:26 sent:26 3.5( v 25'732 (0'0,25'732] local-lis/les=24/25 n=732 ec=15/15 lis/c=24/19 les/c/f=25/20/0 sis=26 pruub=12.015850067s) ([19,25] all_participants={1, 2, 3} intervals=[([24,25] acting={2, 3})])) features: 0x3f03cffffffdffff +create_info", 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.222 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.124466+0000" 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.124470+0000" 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.223 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.124581+0000" 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.224 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.124699+0000" 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.127772+0000" 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.129522+0000" 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.129863+0000" 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.229 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16777255, 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "0", 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.f", 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MQuery 3.f from 0 query_epoch 26 query: query(info 0'0 epoch_sent 26)", 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.125586+0000" 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.125591+0000" 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.125677+0000" 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.232 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.125831+0000" 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.129243+0000" 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.237 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16777256, 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "0", 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.2", 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MNotifyRec 3.2 from 0 notify: (query:26 sent:26 3.2( v 25'754 (0'0,25'754] local-lis/les=24/25 n=754 ec=15/15 lis/c=24/19 les/c/f=25/20/0 sis=26 pruub=11.970928192s) ([19,25] all_participants={0, 1, 3} intervals=[([24,25] acting={0, 3})])) features: 0x3f03cffffffdffff +create_info", 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.129610+0000" 2024-09-22T03:12:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.129613+0000" 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.129693+0000" 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.129775+0000" 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.130651+0000", 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PerShardPipeline::create_or_wait_pg", 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 16777255 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.242 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.245 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 37, 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.094850+0000" 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.246 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.094858+0000" 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.095034+0000" 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.248 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.095333+0000" 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096267+0000" 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096653+0000" 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.251 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096773+0000" 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.096851+0000", 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PGActivationBlocker", 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.8" 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.253 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554473, 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2", 2024-09-22T03:12:30.260 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "1.6", 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MNotifyRec 1.6 from 2 notify: (query:26 sent:26 1.6( empty local-lis/les=24/25 n=0 ec=11/11 lis/c=24/19 les/c/f=25/20/0 sis=26 pruub=12.014983177s) ([19,25] all_participants={1, 2, 3} intervals=[([24,25] acting={2, 3})])) features: 0x3f03cffffffdffff +create_info", 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.127201+0000" 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.127204+0000" 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.127295+0000" 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.127358+0000" 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.127789+0000" 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.128366+0000" 2024-09-22T03:12:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.128405+0000" 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.267 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16777259, 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "0", 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "1.7", 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.268 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MNotifyRec 1.7 from 0 notify: (query:26 sent:26 1.7( empty local-lis/les=24/25 n=0 ec=11/11 lis/c=24/19 les/c/f=25/20/0 sis=26 pruub=11.966699600s) ([19,25] all_participants={0, 1, 3} intervals=[([24,25] acting={0, 3})])) features: 0x3f03cffffffdffff +create_info", 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132223+0000" 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132226+0000" 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132301+0000" 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132498+0000" 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132553+0000" 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132643+0000" 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132706+0000" 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.275 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554474, 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2", 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "1.6", 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) +create_info", 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.276 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.131974+0000" 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.131977+0000" 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132038+0000" 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132097+0000" 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132369+0000" 2024-09-22T03:12:30.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.280 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132466+0000" 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.132571+0000", 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.282 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 33554473 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.283 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16777260, 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "0", 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "1.7", 2024-09-22T03:12:30.284 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 26, 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 26, 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 26 epoch_requested: 26 MLogRec from 0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) +create_info", 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.133378+0000" 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.133381+0000" 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.285 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.133472+0000" 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.133528+0000" 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.133587+0000" 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.133758+0000" 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:12:30.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.133796+0000", 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 16777259 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.292 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 40, 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098192+0000" 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098196+0000" 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098325+0000" 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098420+0000" 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.295 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098647+0000" 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098881+0000" 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.098978+0000" 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.099028+0000", 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PGActivationBlocker", 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "3.d" 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.302 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.303 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.304 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 41, 2024-09-22T03:12:30.306 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.099205+0000" 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.099212+0000" 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.099358+0000" 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.099482+0000" 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.100173+0000" 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.100383+0000" 2024-09-22T03:12:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:12:30.100518+0000", 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 40 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.317 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.318 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:12:30.320 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.321 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.321 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:12:30.321 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:12:30.321 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:12:30.322 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:12:30.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:30.421+0000 7fe276e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:30.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:30.429+0000 7fe276e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:30.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 75 16 11693 11677 9.72972 14.3125 0.0209583 0.102145 2024-09-22T03:12:31.010 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:12:31.028 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:12:31.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:31.172+0000 7fca0866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:31.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:31.181+0000 7fca0866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:31.385 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:31.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 76 16 11693 11677 9.6017 0 - 0.102145 2024-09-22T03:12:31.488 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:31.485+0000 7f1354b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:31.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:31.489+0000 7f1354b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:32.014 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:12:32.033 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:12:32.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:32.162+0000 7fdb306b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:32.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:32.172+0000 7fdb306b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:32.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 77 16 11693 11677 9.47701 0 - 0.102145 2024-09-22T03:12:33.014 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:12:33.038 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:12:33.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:33.187+0000 7f8ac58b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:33.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:33.199+0000 7f8ac58b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:33.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 78 16 11937 11921 9.551 5.08333 0.0246446 0.10465 2024-09-22T03:12:33.560 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:33.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:33.712+0000 7f9754517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:33.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:33.721+0000 7f97532b5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:34.024 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:12:34.046 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:12:34.046 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:34.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:34.190+0000 7f3e41797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:34.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:34.200+0000 7f3e41797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:34.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 79 16 12232 12216 9.66347 18.4375 0.0615712 0.103446 2024-09-22T03:12:34.510 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:34.510 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:34.530 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-22T03:12:34.025956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":773980723,"num_objects":11805,"num_object_clones":0,"num_object_copies":35415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11940,"num_write_kb":757160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11891,"ondisk_log_size":11891,"up":72,"acting":60,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":250508,"kb_used_data":109836,"kb_used_omap":0,"kb_used_meta":140608,"kb_avail":377236852,"statfs":{"total":386547056640,"available":386290536448,"internally_reserved":0,"allocated":112472064,"data_stored":1784449538,"data_compressed":84723648,"data_compressed_allocated":111521792,"data_compressed_original":1784102912,"omap_allocated":0,"internal_metadata":143982592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61276160,"num_objects":935,"num_object_clones":0,"num_object_copies":2805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022412"},"pg_stats":[{"pgid":"3.f","version":"25'763","reported_seq":797,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.108318+0000","last_change":"2024-09-22T03:12:32.139030+0000","last_active":"2024-09-22T03:12:30.052738+0000","last_peered":"2024-09-22T03:12:29.928207+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:10.096291+0000","last_became_peered":"2024-09-22T03:12:10.096291+0000","last_unstale":"2024-09-22T03:12:32.108318+0000","last_undegraded":"2024-09-22T03:12:32.108318+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50003968,"num_objects":763,"num_object_clones":0,"num_object_copies":2289,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":763,"num_write_kb":48832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"28'764","reported_seq":808,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.049626+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:33.049626+0000","last_peered":"2024-09-22T03:12:33.049626+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:33.049626+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50135040,"num_objects":765,"num_object_clones":0,"num_object_copies":2295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":48960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":230}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'733","reported_seq":762,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.142159+0000","last_change":"2024-09-22T03:12:32.166066+0000","last_active":"2024-09-22T03:12:31.032321+0000","last_peered":"2024-09-22T03:12:30.017449+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:10.087609+0000","last_became_peered":"2024-09-22T03:12:10.087609+0000","last_unstale":"2024-09-22T03:12:32.142159+0000","last_undegraded":"2024-09-22T03:12:32.142159+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48037888,"num_objects":733,"num_object_clones":0,"num_object_copies":2199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":46912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"25'760","reported_seq":805,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.046355+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:32.046355+0000","last_peered":"2024-09-22T03:12:32.046355+0000","last_clean":"2024-09-22T03:12:32.046355+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:32.046355+0000","last_undegraded":"2024-09-22T03:12:32.046355+0000","last_fullsized":"2024-09-22T03:12:32.046355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49807360,"num_objects":760,"num_object_clones":0,"num_object_copies":2280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":48640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"29'733","reported_seq":782,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.238628+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:33.238628+0000","last_peered":"2024-09-22T03:12:33.238628+0000","last_clean":"2024-09-22T03:12:33.238628+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:33.238628+0000","last_undegraded":"2024-09-22T03:12:33.238628+0000","last_fullsized":"2024-09-22T03:12:33.238628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48103424,"num_objects":734,"num_object_clones":0,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":734,"num_write_kb":46976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'719","reported_seq":754,"reported_epoch":28,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:32.106233+0000","last_change":"2024-09-22T03:12:32.181025+0000","last_active":"2024-09-22T03:12:30.044669+0000","last_peered":"2024-09-22T03:12:30.025664+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:10.069384+0000","last_became_peered":"2024-09-22T03:12:10.069384+0000","last_unstale":"2024-09-22T03:12:32.106233+0000","last_undegraded":"2024-09-22T03:12:32.106233+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'720","reported_seq":755,"reported_epoch":28,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:32.044280+0000","last_change":"2024-09-22T03:12:32.184843+0000","last_active":"2024-09-22T03:12:30.041759+0000","last_peered":"2024-09-22T03:12:29.950853+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:10.065362+0000","last_became_peered":"2024-09-22T03:12:10.065362+0000","last_unstale":"2024-09-22T03:12:32.044280+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47185920,"num_objects":720,"num_object_clones":0,"num_object_copies":2160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":46080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":253}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"29'749","reported_seq":798,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.215962+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:33.215962+0000","last_peered":"2024-09-22T03:12:33.215962+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:33.215962+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152000,"num_objects":750,"num_object_clones":0,"num_object_copies":2250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":750,"num_write_kb":48000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":250}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.050226+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:33.050226+0000","last_peered":"2024-09-22T03:12:33.050226+0000","last_clean":"2024-09-22T03:12:33.050226+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:33.050226+0000","last_undegraded":"2024-09-22T03:12:33.050226+0000","last_fullsized":"2024-09-22T03:12:33.050226+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'754","reported_seq":783,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.046866+0000","last_change":"2024-09-22T03:12:32.076673+0000","last_active":"2024-09-22T03:12:31.031375+0000","last_peered":"2024-09-22T03:12:29.963927+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:10.082380+0000","last_became_peered":"2024-09-22T03:12:10.082380+0000","last_unstale":"2024-09-22T03:12:32.046866+0000","last_undegraded":"2024-09-22T03:12:32.046866+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49414144,"num_objects":754,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":754,"num_write_kb":48256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.022047+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:32.022047+0000","last_peered":"2024-09-22T03:12:32.022047+0000","last_clean":"2024-09-22T03:12:32.022047+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:32.022047+0000","last_undegraded":"2024-09-22T03:12:32.022047+0000","last_fullsized":"2024-09-22T03:12:32.022047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"29'732","reported_seq":781,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.172895+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:33.172895+0000","last_peered":"2024-09-22T03:12:33.172895+0000","last_clean":"2024-09-22T03:12:33.172895+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:33.172895+0000","last_undegraded":"2024-09-22T03:12:33.172895+0000","last_fullsized":"2024-09-22T03:12:33.172895+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48037888,"num_objects":733,"num_object_clones":0,"num_object_copies":2199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":46912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.040905+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:33.040905+0000","last_peered":"2024-09-22T03:12:33.040905+0000","last_clean":"2024-09-22T03:12:33.040905+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:33.040905+0000","last_undegraded":"2024-09-22T03:12:33.040905+0000","last_fullsized":"2024-09-22T03:12:33.040905+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"29'735","reported_seq":782,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.078793+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:33.078793+0000","last_peered":"2024-09-22T03:12:33.078793+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:33.078793+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":244}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.051278+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:33.051278+0000","last_peered":"2024-09-22T03:12:33.051278+0000","last_clean":"2024-09-22T03:12:33.051278+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:33.051278+0000","last_undegraded":"2024-09-22T03:12:33.051278+0000","last_fullsized":"2024-09-22T03:12:33.051278+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":189,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.046055+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:32.046055+0000","last_peered":"2024-09-22T03:12:32.046055+0000","last_clean":"2024-09-22T03:12:32.046055+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:32.046055+0000","last_undegraded":"2024-09-22T03:12:32.046055+0000","last_fullsized":"2024-09-22T03:12:32.046055+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"29'709","reported_seq":751,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.195303+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:33.195303+0000","last_peered":"2024-09-22T03:12:33.195303+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:33.195303+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46530560,"num_objects":710,"num_object_clones":0,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":710,"num_write_kb":45440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":238}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"28'728","reported_seq":777,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.044730+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:33.044730+0000","last_peered":"2024-09-22T03:12:33.044730+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:33.044730+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47775744,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":46656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":228}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.074574+0000","last_change":"2024-09-22T03:12:10.074230+0000","last_active":"2024-09-22T03:12:10.074574+0000","last_peered":"2024-09-22T03:12:10.074574+0000","last_clean":"2024-09-22T03:11:06.315471+0000","last_became_active":"2024-09-22T03:12:10.074230+0000","last_became_peered":"2024-09-22T03:12:10.074230+0000","last_unstale":"2024-09-22T03:12:10.074574+0000","last_undegraded":"2024-09-22T03:12:10.074574+0000","last_fullsized":"2024-09-22T03:12:10.032035+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"29'718","reported_seq":765,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.157758+0000","last_change":"2024-09-22T03:12:32.629406+0000","last_active":"2024-09-22T03:12:33.157758+0000","last_peered":"2024-09-22T03:12:33.157758+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:33.157758+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":239}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.049152+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:33.049152+0000","last_peered":"2024-09-22T03:12:33.049152+0000","last_clean":"2024-09-22T03:12:33.049152+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:33.049152+0000","last_undegraded":"2024-09-22T03:12:33.049152+0000","last_fullsized":"2024-09-22T03:12:33.049152+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"29'729","reported_seq":778,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.206212+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:33.206212+0000","last_peered":"2024-09-22T03:12:33.206212+0000","last_clean":"2024-09-22T03:12:33.206212+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:33.206212+0000","last_undegraded":"2024-09-22T03:12:33.206212+0000","last_fullsized":"2024-09-22T03:12:33.206212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":730,"num_write_kb":46720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.113020+0000","last_change":"2024-09-22T03:12:10.112365+0000","last_active":"2024-09-22T03:12:10.113020+0000","last_peered":"2024-09-22T03:12:10.113020+0000","last_clean":"2024-09-22T03:11:06.340280+0000","last_became_active":"2024-09-22T03:12:10.112365+0000","last_became_peered":"2024-09-22T03:12:10.112365+0000","last_unstale":"2024-09-22T03:12:10.113020+0000","last_undegraded":"2024-09-22T03:12:10.113020+0000","last_fullsized":"2024-09-22T03:12:10.045988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"29'745","reported_seq":798,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.205025+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:33.205025+0000","last_peered":"2024-09-22T03:12:33.205025+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:33.205025+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.083082+0000","last_change":"2024-09-22T03:12:10.082403+0000","last_active":"2024-09-22T03:12:10.083082+0000","last_peered":"2024-09-22T03:12:10.083082+0000","last_clean":"2024-09-22T03:11:06.304946+0000","last_became_active":"2024-09-22T03:12:10.082403+0000","last_became_peered":"2024-09-22T03:12:10.082403+0000","last_unstale":"2024-09-22T03:12:10.083082+0000","last_undegraded":"2024-09-22T03:12:10.083082+0000","last_fullsized":"2024-09-22T03:12:10.037461+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":773390336,"num_objects":11801,"num_object_clones":0,"num_object_copies":35403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11801,"num_write_kb":755264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":113713152,"data_stored":1819410432,"data_compressed":86392012,"data_compressed_allocated":113713152,"data_compressed_original":1819410432,"omap_allocated":0,"internal_metadata":0},"log_size":11791,"ondisk_log_size":11791,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":21,"acting":21,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72208,"kb_used_data":41216,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94299632,"statfs":{"total":96636764160,"available":96562823168,"internally_reserved":0,"allocated":42205184,"data_stored":671562178,"data_compressed":31884590,"data_compressed_allocated":41967616,"data_compressed_original":671481856,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71700,"kb_used_data":32388,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94300140,"statfs":{"total":96636764160,"available":96563343360,"internally_reserved":0,"allocated":33165312,"data_stored":526531554,"data_compressed":24999923,"data_compressed_allocated":32907264,"data_compressed_original":526434304,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":19,"seq":81604378628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34952,"kb_used_data":3896,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94336888,"statfs":{"total":96636764160,"available":96600973312,"internally_reserved":0,"allocated":3989504,"data_stored":60483276,"data_compressed":2870377,"data_compressed_allocated":3780608,"data_compressed_original":60407808,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71648,"kb_used_data":32336,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94300192,"statfs":{"total":96636764160,"available":96563396608,"internally_reserved":0,"allocated":33112064,"data_stored":525872530,"data_compressed":24968758,"data_compressed_allocated":32866304,"data_compressed_original":525778944,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32825344,"data_stored":525205504,"data_compressed":24938786,"data_compressed_allocated":32825344,"data_compressed_original":525205504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33378304,"data_stored":534052864,"data_compressed":25358951,"data_compressed_allocated":33378304,"data_compressed_original":534052864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42631168,"data_stored":682098688,"data_compressed":32388734,"data_compressed_allocated":42631168,"data_compressed_original":682098688,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:34.531 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:34.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:34.680+0000 7fad1750e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:34.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:34.694+0000 7fad1750e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:35.021 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:35.021 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:35.042 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-22T03:12:34.025956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":773980723,"num_objects":11805,"num_object_clones":0,"num_object_copies":35415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11940,"num_write_kb":757160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11891,"ondisk_log_size":11891,"up":72,"acting":60,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":250508,"kb_used_data":109836,"kb_used_omap":0,"kb_used_meta":140608,"kb_avail":377236852,"statfs":{"total":386547056640,"available":386290536448,"internally_reserved":0,"allocated":112472064,"data_stored":1784449538,"data_compressed":84723648,"data_compressed_allocated":111521792,"data_compressed_original":1784102912,"omap_allocated":0,"internal_metadata":143982592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61276160,"num_objects":935,"num_object_clones":0,"num_object_copies":2805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022412"},"pg_stats":[{"pgid":"3.f","version":"25'763","reported_seq":797,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.108318+0000","last_change":"2024-09-22T03:12:32.139030+0000","last_active":"2024-09-22T03:12:30.052738+0000","last_peered":"2024-09-22T03:12:29.928207+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:10.096291+0000","last_became_peered":"2024-09-22T03:12:10.096291+0000","last_unstale":"2024-09-22T03:12:32.108318+0000","last_undegraded":"2024-09-22T03:12:32.108318+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50003968,"num_objects":763,"num_object_clones":0,"num_object_copies":2289,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":763,"num_write_kb":48832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"28'764","reported_seq":808,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.049626+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:33.049626+0000","last_peered":"2024-09-22T03:12:33.049626+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:33.049626+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50135040,"num_objects":765,"num_object_clones":0,"num_object_copies":2295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":48960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":230}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'733","reported_seq":762,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.142159+0000","last_change":"2024-09-22T03:12:32.166066+0000","last_active":"2024-09-22T03:12:31.032321+0000","last_peered":"2024-09-22T03:12:30.017449+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:10.087609+0000","last_became_peered":"2024-09-22T03:12:10.087609+0000","last_unstale":"2024-09-22T03:12:32.142159+0000","last_undegraded":"2024-09-22T03:12:32.142159+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48037888,"num_objects":733,"num_object_clones":0,"num_object_copies":2199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":46912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"25'760","reported_seq":805,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.046355+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:32.046355+0000","last_peered":"2024-09-22T03:12:32.046355+0000","last_clean":"2024-09-22T03:12:32.046355+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:32.046355+0000","last_undegraded":"2024-09-22T03:12:32.046355+0000","last_fullsized":"2024-09-22T03:12:32.046355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49807360,"num_objects":760,"num_object_clones":0,"num_object_copies":2280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":48640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"29'733","reported_seq":782,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.238628+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:33.238628+0000","last_peered":"2024-09-22T03:12:33.238628+0000","last_clean":"2024-09-22T03:12:33.238628+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:33.238628+0000","last_undegraded":"2024-09-22T03:12:33.238628+0000","last_fullsized":"2024-09-22T03:12:33.238628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48103424,"num_objects":734,"num_object_clones":0,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":734,"num_write_kb":46976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'719","reported_seq":754,"reported_epoch":28,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:32.106233+0000","last_change":"2024-09-22T03:12:32.181025+0000","last_active":"2024-09-22T03:12:30.044669+0000","last_peered":"2024-09-22T03:12:30.025664+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:10.069384+0000","last_became_peered":"2024-09-22T03:12:10.069384+0000","last_unstale":"2024-09-22T03:12:32.106233+0000","last_undegraded":"2024-09-22T03:12:32.106233+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'720","reported_seq":755,"reported_epoch":28,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:32.044280+0000","last_change":"2024-09-22T03:12:32.184843+0000","last_active":"2024-09-22T03:12:30.041759+0000","last_peered":"2024-09-22T03:12:29.950853+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:10.065362+0000","last_became_peered":"2024-09-22T03:12:10.065362+0000","last_unstale":"2024-09-22T03:12:32.044280+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47185920,"num_objects":720,"num_object_clones":0,"num_object_copies":2160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":46080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":253}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"29'749","reported_seq":798,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.215962+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:33.215962+0000","last_peered":"2024-09-22T03:12:33.215962+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:33.215962+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152000,"num_objects":750,"num_object_clones":0,"num_object_copies":2250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":750,"num_write_kb":48000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":250}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.050226+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:33.050226+0000","last_peered":"2024-09-22T03:12:33.050226+0000","last_clean":"2024-09-22T03:12:33.050226+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:33.050226+0000","last_undegraded":"2024-09-22T03:12:33.050226+0000","last_fullsized":"2024-09-22T03:12:33.050226+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'754","reported_seq":783,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.046866+0000","last_change":"2024-09-22T03:12:32.076673+0000","last_active":"2024-09-22T03:12:31.031375+0000","last_peered":"2024-09-22T03:12:29.963927+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:10.082380+0000","last_became_peered":"2024-09-22T03:12:10.082380+0000","last_unstale":"2024-09-22T03:12:32.046866+0000","last_undegraded":"2024-09-22T03:12:32.046866+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49414144,"num_objects":754,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":754,"num_write_kb":48256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.022047+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:32.022047+0000","last_peered":"2024-09-22T03:12:32.022047+0000","last_clean":"2024-09-22T03:12:32.022047+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:32.022047+0000","last_undegraded":"2024-09-22T03:12:32.022047+0000","last_fullsized":"2024-09-22T03:12:32.022047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"29'732","reported_seq":781,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.172895+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:33.172895+0000","last_peered":"2024-09-22T03:12:33.172895+0000","last_clean":"2024-09-22T03:12:33.172895+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:33.172895+0000","last_undegraded":"2024-09-22T03:12:33.172895+0000","last_fullsized":"2024-09-22T03:12:33.172895+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48037888,"num_objects":733,"num_object_clones":0,"num_object_copies":2199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":46912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.040905+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:33.040905+0000","last_peered":"2024-09-22T03:12:33.040905+0000","last_clean":"2024-09-22T03:12:33.040905+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:33.040905+0000","last_undegraded":"2024-09-22T03:12:33.040905+0000","last_fullsized":"2024-09-22T03:12:33.040905+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"29'735","reported_seq":782,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.078793+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:33.078793+0000","last_peered":"2024-09-22T03:12:33.078793+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:33.078793+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":244}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.051278+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:33.051278+0000","last_peered":"2024-09-22T03:12:33.051278+0000","last_clean":"2024-09-22T03:12:33.051278+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:33.051278+0000","last_undegraded":"2024-09-22T03:12:33.051278+0000","last_fullsized":"2024-09-22T03:12:33.051278+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":189,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.046055+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:32.046055+0000","last_peered":"2024-09-22T03:12:32.046055+0000","last_clean":"2024-09-22T03:12:32.046055+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:32.046055+0000","last_undegraded":"2024-09-22T03:12:32.046055+0000","last_fullsized":"2024-09-22T03:12:32.046055+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"29'709","reported_seq":751,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.195303+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:33.195303+0000","last_peered":"2024-09-22T03:12:33.195303+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:33.195303+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46530560,"num_objects":710,"num_object_clones":0,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":710,"num_write_kb":45440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":238}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"28'728","reported_seq":777,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.044730+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:33.044730+0000","last_peered":"2024-09-22T03:12:33.044730+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:33.044730+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47775744,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":46656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":228}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.074574+0000","last_change":"2024-09-22T03:12:10.074230+0000","last_active":"2024-09-22T03:12:10.074574+0000","last_peered":"2024-09-22T03:12:10.074574+0000","last_clean":"2024-09-22T03:11:06.315471+0000","last_became_active":"2024-09-22T03:12:10.074230+0000","last_became_peered":"2024-09-22T03:12:10.074230+0000","last_unstale":"2024-09-22T03:12:10.074574+0000","last_undegraded":"2024-09-22T03:12:10.074574+0000","last_fullsized":"2024-09-22T03:12:10.032035+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"29'718","reported_seq":765,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.157758+0000","last_change":"2024-09-22T03:12:32.629406+0000","last_active":"2024-09-22T03:12:33.157758+0000","last_peered":"2024-09-22T03:12:33.157758+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:33.157758+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":239}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.049152+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:33.049152+0000","last_peered":"2024-09-22T03:12:33.049152+0000","last_clean":"2024-09-22T03:12:33.049152+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:33.049152+0000","last_undegraded":"2024-09-22T03:12:33.049152+0000","last_fullsized":"2024-09-22T03:12:33.049152+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"29'729","reported_seq":778,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.206212+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:33.206212+0000","last_peered":"2024-09-22T03:12:33.206212+0000","last_clean":"2024-09-22T03:12:33.206212+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:33.206212+0000","last_undegraded":"2024-09-22T03:12:33.206212+0000","last_fullsized":"2024-09-22T03:12:33.206212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":730,"num_write_kb":46720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.113020+0000","last_change":"2024-09-22T03:12:10.112365+0000","last_active":"2024-09-22T03:12:10.113020+0000","last_peered":"2024-09-22T03:12:10.113020+0000","last_clean":"2024-09-22T03:11:06.340280+0000","last_became_active":"2024-09-22T03:12:10.112365+0000","last_became_peered":"2024-09-22T03:12:10.112365+0000","last_unstale":"2024-09-22T03:12:10.113020+0000","last_undegraded":"2024-09-22T03:12:10.113020+0000","last_fullsized":"2024-09-22T03:12:10.045988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"29'745","reported_seq":798,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.205025+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:33.205025+0000","last_peered":"2024-09-22T03:12:33.205025+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:33.205025+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.083082+0000","last_change":"2024-09-22T03:12:10.082403+0000","last_active":"2024-09-22T03:12:10.083082+0000","last_peered":"2024-09-22T03:12:10.083082+0000","last_clean":"2024-09-22T03:11:06.304946+0000","last_became_active":"2024-09-22T03:12:10.082403+0000","last_became_peered":"2024-09-22T03:12:10.082403+0000","last_unstale":"2024-09-22T03:12:10.083082+0000","last_undegraded":"2024-09-22T03:12:10.083082+0000","last_fullsized":"2024-09-22T03:12:10.037461+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":773390336,"num_objects":11801,"num_object_clones":0,"num_object_copies":35403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11801,"num_write_kb":755264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":113713152,"data_stored":1819410432,"data_compressed":86392012,"data_compressed_allocated":113713152,"data_compressed_original":1819410432,"omap_allocated":0,"internal_metadata":0},"log_size":11791,"ondisk_log_size":11791,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":21,"acting":21,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72208,"kb_used_data":41216,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94299632,"statfs":{"total":96636764160,"available":96562823168,"internally_reserved":0,"allocated":42205184,"data_stored":671562178,"data_compressed":31884590,"data_compressed_allocated":41967616,"data_compressed_original":671481856,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71700,"kb_used_data":32388,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94300140,"statfs":{"total":96636764160,"available":96563343360,"internally_reserved":0,"allocated":33165312,"data_stored":526531554,"data_compressed":24999923,"data_compressed_allocated":32907264,"data_compressed_original":526434304,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":19,"seq":81604378628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34952,"kb_used_data":3896,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94336888,"statfs":{"total":96636764160,"available":96600973312,"internally_reserved":0,"allocated":3989504,"data_stored":60483276,"data_compressed":2870377,"data_compressed_allocated":3780608,"data_compressed_original":60407808,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71648,"kb_used_data":32336,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94300192,"statfs":{"total":96636764160,"available":96563396608,"internally_reserved":0,"allocated":33112064,"data_stored":525872530,"data_compressed":24968758,"data_compressed_allocated":32866304,"data_compressed_original":525778944,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32825344,"data_stored":525205504,"data_compressed":24938786,"data_compressed_allocated":32825344,"data_compressed_original":525205504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33378304,"data_stored":534052864,"data_compressed":25358951,"data_compressed_allocated":33378304,"data_compressed_original":534052864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42631168,"data_stored":682098688,"data_compressed":32388734,"data_compressed_allocated":42631168,"data_compressed_original":682098688,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:35.044 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:12:35.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:35.202+0000 7f53b2397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:35.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:35.211+0000 7f53b2397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:35.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:12:35.484560+0000 min lat: 0.00921296 max lat: 31.8821 avg lat: 0.102154 2024-09-22T03:12:35.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:12:35.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 80 16 12541 12525 9.78405 19.3125 0.0508246 0.102154 2024-09-22T03:12:35.646 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:35.646 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7254/33774 objects degraded (21.478%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":100,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":30,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+undersized+degraded","count":9},{"state_name":"active+clean","count":8},{"state_name":"active+undersized","count":3},{"state_name":"activating+undersized+remapped","count":3},{"state_name":"activating+undersized+degraded+remapped","count":2}],"num_pgs":25,"num_pools":3,"num_objects":11258,"data_bytes":738132531,"bytes_used":245358592,"bytes_avail":386301698048,"bytes_total":386547056640,"inactive_pgs_ratio":0.20000000298023224,"degraded_objects":7254,"degraded_total":33774,"degraded_ratio":0.21478060046189376,"write_bytes_sec":6355218,"read_op_per_sec":0,"write_op_per_sec":96},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:12:35.665 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-22T03:12:35.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:35.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:35.810+0000 7f17c3117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:35.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:35.822+0000 7f17c3117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:35.942 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:36.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:36.090+0000 7f75c26c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:36.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:36.100+0000 7f75c26c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:36.156 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:36.156 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:36.176 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-22T03:12:35.241360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":773980723,"num_objects":11805,"num_object_clones":0,"num_object_copies":35415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11940,"num_write_kb":757160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11891,"ondisk_log_size":11891,"up":72,"acting":60,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":250508,"kb_used_data":109836,"kb_used_omap":0,"kb_used_meta":140608,"kb_avail":377236852,"statfs":{"total":386547056640,"available":386290536448,"internally_reserved":0,"allocated":112472064,"data_stored":1784449538,"data_compressed":84723648,"data_compressed_allocated":111521792,"data_compressed_original":1784102912,"omap_allocated":0,"internal_metadata":143982592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61276160,"num_objects":935,"num_object_clones":0,"num_object_copies":2805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.231259"},"pg_stats":[{"pgid":"3.f","version":"25'763","reported_seq":797,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.108318+0000","last_change":"2024-09-22T03:12:32.139030+0000","last_active":"2024-09-22T03:12:30.052738+0000","last_peered":"2024-09-22T03:12:29.928207+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:10.096291+0000","last_became_peered":"2024-09-22T03:12:10.096291+0000","last_unstale":"2024-09-22T03:12:32.108318+0000","last_undegraded":"2024-09-22T03:12:32.108318+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50003968,"num_objects":763,"num_object_clones":0,"num_object_copies":2289,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":763,"num_write_kb":48832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"28'764","reported_seq":808,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.049626+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:33.049626+0000","last_peered":"2024-09-22T03:12:33.049626+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:33.049626+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50135040,"num_objects":765,"num_object_clones":0,"num_object_copies":2295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":48960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":230}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'733","reported_seq":762,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.142159+0000","last_change":"2024-09-22T03:12:32.166066+0000","last_active":"2024-09-22T03:12:31.032321+0000","last_peered":"2024-09-22T03:12:30.017449+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:10.087609+0000","last_became_peered":"2024-09-22T03:12:10.087609+0000","last_unstale":"2024-09-22T03:12:32.142159+0000","last_undegraded":"2024-09-22T03:12:32.142159+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48037888,"num_objects":733,"num_object_clones":0,"num_object_copies":2199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":46912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"25'760","reported_seq":805,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.046355+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:32.046355+0000","last_peered":"2024-09-22T03:12:32.046355+0000","last_clean":"2024-09-22T03:12:32.046355+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:32.046355+0000","last_undegraded":"2024-09-22T03:12:32.046355+0000","last_fullsized":"2024-09-22T03:12:32.046355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49807360,"num_objects":760,"num_object_clones":0,"num_object_copies":2280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":48640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"29'733","reported_seq":782,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.238628+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:33.238628+0000","last_peered":"2024-09-22T03:12:33.238628+0000","last_clean":"2024-09-22T03:12:33.238628+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:33.238628+0000","last_undegraded":"2024-09-22T03:12:33.238628+0000","last_fullsized":"2024-09-22T03:12:33.238628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48103424,"num_objects":734,"num_object_clones":0,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":734,"num_write_kb":46976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'719","reported_seq":754,"reported_epoch":28,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:32.106233+0000","last_change":"2024-09-22T03:12:32.181025+0000","last_active":"2024-09-22T03:12:30.044669+0000","last_peered":"2024-09-22T03:12:30.025664+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:10.069384+0000","last_became_peered":"2024-09-22T03:12:10.069384+0000","last_unstale":"2024-09-22T03:12:32.106233+0000","last_undegraded":"2024-09-22T03:12:32.106233+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'720","reported_seq":755,"reported_epoch":28,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:32.044280+0000","last_change":"2024-09-22T03:12:32.184843+0000","last_active":"2024-09-22T03:12:30.041759+0000","last_peered":"2024-09-22T03:12:29.950853+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:10.065362+0000","last_became_peered":"2024-09-22T03:12:10.065362+0000","last_unstale":"2024-09-22T03:12:32.044280+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47185920,"num_objects":720,"num_object_clones":0,"num_object_copies":2160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":46080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":253}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"29'749","reported_seq":798,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.215962+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:33.215962+0000","last_peered":"2024-09-22T03:12:33.215962+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:33.215962+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152000,"num_objects":750,"num_object_clones":0,"num_object_copies":2250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":750,"num_write_kb":48000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":250}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.050226+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:33.050226+0000","last_peered":"2024-09-22T03:12:33.050226+0000","last_clean":"2024-09-22T03:12:33.050226+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:33.050226+0000","last_undegraded":"2024-09-22T03:12:33.050226+0000","last_fullsized":"2024-09-22T03:12:33.050226+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'754","reported_seq":783,"reported_epoch":28,"state":"activating+undersized+remapped","last_fresh":"2024-09-22T03:12:32.046866+0000","last_change":"2024-09-22T03:12:32.076673+0000","last_active":"2024-09-22T03:12:31.031375+0000","last_peered":"2024-09-22T03:12:29.963927+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:10.082380+0000","last_became_peered":"2024-09-22T03:12:10.082380+0000","last_unstale":"2024-09-22T03:12:32.046866+0000","last_undegraded":"2024-09-22T03:12:32.046866+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49414144,"num_objects":754,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":754,"num_write_kb":48256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.022047+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:32.022047+0000","last_peered":"2024-09-22T03:12:32.022047+0000","last_clean":"2024-09-22T03:12:32.022047+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:32.022047+0000","last_undegraded":"2024-09-22T03:12:32.022047+0000","last_fullsized":"2024-09-22T03:12:32.022047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"29'732","reported_seq":781,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.172895+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:33.172895+0000","last_peered":"2024-09-22T03:12:33.172895+0000","last_clean":"2024-09-22T03:12:33.172895+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:33.172895+0000","last_undegraded":"2024-09-22T03:12:33.172895+0000","last_fullsized":"2024-09-22T03:12:33.172895+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48037888,"num_objects":733,"num_object_clones":0,"num_object_copies":2199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":46912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.040905+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:33.040905+0000","last_peered":"2024-09-22T03:12:33.040905+0000","last_clean":"2024-09-22T03:12:33.040905+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:33.040905+0000","last_undegraded":"2024-09-22T03:12:33.040905+0000","last_fullsized":"2024-09-22T03:12:33.040905+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"29'735","reported_seq":782,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.078793+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:33.078793+0000","last_peered":"2024-09-22T03:12:33.078793+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:33.078793+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":244}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.051278+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:33.051278+0000","last_peered":"2024-09-22T03:12:33.051278+0000","last_clean":"2024-09-22T03:12:33.051278+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:33.051278+0000","last_undegraded":"2024-09-22T03:12:33.051278+0000","last_fullsized":"2024-09-22T03:12:33.051278+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":189,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-22T03:12:32.046055+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:32.046055+0000","last_peered":"2024-09-22T03:12:32.046055+0000","last_clean":"2024-09-22T03:12:32.046055+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:32.046055+0000","last_undegraded":"2024-09-22T03:12:32.046055+0000","last_fullsized":"2024-09-22T03:12:32.046055+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"29'709","reported_seq":751,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.195303+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:33.195303+0000","last_peered":"2024-09-22T03:12:33.195303+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:33.195303+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46530560,"num_objects":710,"num_object_clones":0,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":710,"num_write_kb":45440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":238}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"28'728","reported_seq":777,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.044730+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:33.044730+0000","last_peered":"2024-09-22T03:12:33.044730+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:33.044730+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47775744,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":46656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":228}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.074574+0000","last_change":"2024-09-22T03:12:10.074230+0000","last_active":"2024-09-22T03:12:10.074574+0000","last_peered":"2024-09-22T03:12:10.074574+0000","last_clean":"2024-09-22T03:11:06.315471+0000","last_became_active":"2024-09-22T03:12:10.074230+0000","last_became_peered":"2024-09-22T03:12:10.074230+0000","last_unstale":"2024-09-22T03:12:10.074574+0000","last_undegraded":"2024-09-22T03:12:10.074574+0000","last_fullsized":"2024-09-22T03:12:10.032035+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"29'718","reported_seq":765,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.157758+0000","last_change":"2024-09-22T03:12:32.629406+0000","last_active":"2024-09-22T03:12:33.157758+0000","last_peered":"2024-09-22T03:12:33.157758+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:33.157758+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":239}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.049152+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:33.049152+0000","last_peered":"2024-09-22T03:12:33.049152+0000","last_clean":"2024-09-22T03:12:33.049152+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:33.049152+0000","last_undegraded":"2024-09-22T03:12:33.049152+0000","last_fullsized":"2024-09-22T03:12:33.049152+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"29'729","reported_seq":778,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-22T03:12:33.206212+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:33.206212+0000","last_peered":"2024-09-22T03:12:33.206212+0000","last_clean":"2024-09-22T03:12:33.206212+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:33.206212+0000","last_undegraded":"2024-09-22T03:12:33.206212+0000","last_fullsized":"2024-09-22T03:12:33.206212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":730,"num_write_kb":46720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.113020+0000","last_change":"2024-09-22T03:12:10.112365+0000","last_active":"2024-09-22T03:12:10.113020+0000","last_peered":"2024-09-22T03:12:10.113020+0000","last_clean":"2024-09-22T03:11:06.340280+0000","last_became_active":"2024-09-22T03:12:10.112365+0000","last_became_peered":"2024-09-22T03:12:10.112365+0000","last_unstale":"2024-09-22T03:12:10.113020+0000","last_undegraded":"2024-09-22T03:12:10.113020+0000","last_fullsized":"2024-09-22T03:12:10.045988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"29'745","reported_seq":798,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:33.205025+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:33.205025+0000","last_peered":"2024-09-22T03:12:33.205025+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:33.205025+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+undersized","last_fresh":"2024-09-22T03:12:10.083082+0000","last_change":"2024-09-22T03:12:10.082403+0000","last_active":"2024-09-22T03:12:10.083082+0000","last_peered":"2024-09-22T03:12:10.083082+0000","last_clean":"2024-09-22T03:11:06.304946+0000","last_became_active":"2024-09-22T03:12:10.082403+0000","last_became_peered":"2024-09-22T03:12:10.082403+0000","last_unstale":"2024-09-22T03:12:10.083082+0000","last_undegraded":"2024-09-22T03:12:10.083082+0000","last_fullsized":"2024-09-22T03:12:10.037461+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":773390336,"num_objects":11801,"num_object_clones":0,"num_object_copies":35403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11801,"num_write_kb":755264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":113713152,"data_stored":1819410432,"data_compressed":86392012,"data_compressed_allocated":113713152,"data_compressed_original":1819410432,"omap_allocated":0,"internal_metadata":0},"log_size":11791,"ondisk_log_size":11791,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":21,"acting":21,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72208,"kb_used_data":41216,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94299632,"statfs":{"total":96636764160,"available":96562823168,"internally_reserved":0,"allocated":42205184,"data_stored":671562178,"data_compressed":31884590,"data_compressed_allocated":41967616,"data_compressed_original":671481856,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71700,"kb_used_data":32388,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94300140,"statfs":{"total":96636764160,"available":96563343360,"internally_reserved":0,"allocated":33165312,"data_stored":526531554,"data_compressed":24999923,"data_compressed_allocated":32907264,"data_compressed_original":526434304,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":19,"seq":81604378628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34952,"kb_used_data":3896,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94336888,"statfs":{"total":96636764160,"available":96600973312,"internally_reserved":0,"allocated":3989504,"data_stored":60483276,"data_compressed":2870377,"data_compressed_allocated":3780608,"data_compressed_original":60407808,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71648,"kb_used_data":32336,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94300192,"statfs":{"total":96636764160,"available":96563396608,"internally_reserved":0,"allocated":33112064,"data_stored":525872530,"data_compressed":24968758,"data_compressed_allocated":32866304,"data_compressed_original":525778944,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32825344,"data_stored":525205504,"data_compressed":24938786,"data_compressed_allocated":32825344,"data_compressed_original":525205504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33378304,"data_stored":534052864,"data_compressed":25358951,"data_compressed_allocated":33378304,"data_compressed_original":534052864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42631168,"data_stored":682098688,"data_compressed":32388734,"data_compressed_allocated":42631168,"data_compressed_original":682098688,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:36.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 81 16 12848 12832 9.90012 19.1875 0.0406967 0.100971 2024-09-22T03:12:37.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 82 16 13151 13135 10.0103 18.9375 0.0384498 0.099854 2024-09-22T03:12:38.342 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:38.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 83 16 13460 13444 10.1224 19.3125 0.0470677 0.0987565 2024-09-22T03:12:38.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:38.490+0000 7f2cd0197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:38.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:38.498+0000 7f2cd0197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:39.178 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:39.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:39.324+0000 7fce7ad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:39.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:39.333+0000 7fce7ad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:39.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 84 16 13760 13744 10.225 18.75 0.066578 0.0977551 2024-09-22T03:12:39.648 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:39.649 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:39.670 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-22T03:12:39.242226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":870318643,"num_objects":13275,"num_object_clones":0,"num_object_copies":39825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13410,"num_write_kb":851240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":18808832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13355,"ondisk_log_size":13355,"up":75,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":278804,"kb_used_data":131988,"kb_used_omap":0,"kb_used_meta":146752,"kb_avail":377208556,"statfs":{"total":386547056640,"available":386261561344,"internally_reserved":0,"allocated":135155712,"data_stored":2145476221,"data_compressed":101864544,"data_compressed_allocated":134082560,"data_compressed_original":2145075200,"omap_allocated":0,"internal_metadata":150274048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":157614080,"num_objects":2405,"num_object_clones":0,"num_object_copies":7215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2405,"num_write_kb":153920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":18808832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.228213"},"pg_stats":[{"pgid":"3.f","version":"30'859","reported_seq":903,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:37.212616+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:37.212616+0000","last_peered":"2024-09-22T03:12:37.212616+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:37.212616+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56360960,"num_objects":860,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":860,"num_write_kb":55040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":325}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"30'853","reported_seq":899,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.005624+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:38.005624+0000","last_peered":"2024-09-22T03:12:38.005624+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:38.005624+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55967744,"num_objects":854,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":854,"num_write_kb":54656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":319}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'806","reported_seq":847,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:36.905037+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:36.905037+0000","last_peered":"2024-09-22T03:12:36.905037+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:36.905037+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52887552,"num_objects":807,"num_object_clones":0,"num_object_copies":2421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":807,"num_write_kb":51648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"30'849","reported_seq":901,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:37.189698+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:37.189698+0000","last_peered":"2024-09-22T03:12:37.189698+0000","last_clean":"2024-09-22T03:12:37.189698+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:37.189698+0000","last_undegraded":"2024-09-22T03:12:37.189698+0000","last_fullsized":"2024-09-22T03:12:37.189698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55705600,"num_objects":850,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":850,"num_write_kb":54400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'836","reported_seq":888,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.230219+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:38.230219+0000","last_peered":"2024-09-22T03:12:38.230219+0000","last_clean":"2024-09-22T03:12:38.230219+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:38.230219+0000","last_undegraded":"2024-09-22T03:12:38.230219+0000","last_fullsized":"2024-09-22T03:12:38.230219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":837,"num_write_kb":53568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"30'800","reported_seq":1132,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:37.236297+0000","last_change":"2024-09-22T03:12:32.631272+0000","last_active":"2024-09-22T03:12:37.236297+0000","last_peered":"2024-09-22T03:12:37.236297+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:32.536732+0000","last_became_peered":"2024-09-22T03:12:32.536732+0000","last_unstale":"2024-09-22T03:12:37.236297+0000","last_undegraded":"2024-09-22T03:12:32.106233+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52494336,"num_objects":801,"num_object_clones":0,"num_object_copies":2403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":801,"num_write_kb":51264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":9371648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"30'800","reported_seq":851,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:37.099249+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:37.099249+0000","last_peered":"2024-09-22T03:12:37.099249+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:37.099249+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52494336,"num_objects":801,"num_object_clones":0,"num_object_copies":2403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":801,"num_write_kb":51264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'862","reported_seq":1033,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.184437+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:38.184437+0000","last_peered":"2024-09-22T03:12:38.184437+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:38.184437+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":863,"num_write_kb":55232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":423}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047905+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:34.047905+0000","last_peered":"2024-09-22T03:12:34.047905+0000","last_clean":"2024-09-22T03:12:34.047905+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:34.047905+0000","last_undegraded":"2024-09-22T03:12:34.047905+0000","last_fullsized":"2024-09-22T03:12:34.047905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"30'841","reported_seq":880,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:36.877609+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:36.877609+0000","last_peered":"2024-09-22T03:12:36.877609+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:36.877609+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55181312,"num_objects":842,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":842,"num_write_kb":53888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.060241+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:34.060241+0000","last_peered":"2024-09-22T03:12:34.060241+0000","last_clean":"2024-09-22T03:12:34.060241+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:34.060241+0000","last_undegraded":"2024-09-22T03:12:34.060241+0000","last_fullsized":"2024-09-22T03:12:34.060241+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'823","reported_seq":875,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.227947+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:38.227947+0000","last_peered":"2024-09-22T03:12:38.227947+0000","last_clean":"2024-09-22T03:12:38.227947+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:38.227947+0000","last_undegraded":"2024-09-22T03:12:38.227947+0000","last_fullsized":"2024-09-22T03:12:38.227947+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54001664,"num_objects":824,"num_object_clones":0,"num_object_copies":2472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":52736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":73,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.043071+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:34.043071+0000","last_peered":"2024-09-22T03:12:34.043071+0000","last_clean":"2024-09-22T03:12:34.043071+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:34.043071+0000","last_undegraded":"2024-09-22T03:12:34.043071+0000","last_fullsized":"2024-09-22T03:12:34.043071+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"30'835","reported_seq":884,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.166335+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:38.166335+0000","last_peered":"2024-09-22T03:12:38.166335+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:38.166335+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54788096,"num_objects":836,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":836,"num_write_kb":53504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":344}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":72,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.072611+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:34.072611+0000","last_peered":"2024-09-22T03:12:34.072611+0000","last_clean":"2024-09-22T03:12:34.072611+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:34.072611+0000","last_undegraded":"2024-09-22T03:12:34.072611+0000","last_fullsized":"2024-09-22T03:12:34.072611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":195,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.067653+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:34.067653+0000","last_peered":"2024-09-22T03:12:34.067653+0000","last_clean":"2024-09-22T03:12:34.067653+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:34.067653+0000","last_undegraded":"2024-09-22T03:12:34.067653+0000","last_fullsized":"2024-09-22T03:12:34.067653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'797","reported_seq":841,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.145900+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:38.145900+0000","last_peered":"2024-09-22T03:12:38.145900+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:38.145900+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52297728,"num_objects":798,"num_object_clones":0,"num_object_copies":2394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":798,"num_write_kb":51072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":326}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"30'830","reported_seq":881,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.169458+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:38.169458+0000","last_peered":"2024-09-22T03:12:38.169458+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:38.169458+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54460416,"num_objects":831,"num_object_clones":0,"num_object_copies":2493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":53184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":330}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.055738+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:34.055738+0000","last_peered":"2024-09-22T03:12:34.055738+0000","last_clean":"2024-09-22T03:12:34.055738+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:34.055738+0000","last_undegraded":"2024-09-22T03:12:34.055738+0000","last_fullsized":"2024-09-22T03:12:34.055738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'807","reported_seq":856,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.154929+0000","last_change":"2024-09-22T03:12:32.629406+0000","last_active":"2024-09-22T03:12:38.154929+0000","last_peered":"2024-09-22T03:12:38.154929+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:38.154929+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52953088,"num_objects":808,"num_object_clones":0,"num_object_copies":2424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":51712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":328}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047474+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:34.047474+0000","last_peered":"2024-09-22T03:12:34.047474+0000","last_clean":"2024-09-22T03:12:34.047474+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:34.047474+0000","last_undegraded":"2024-09-22T03:12:34.047474+0000","last_fullsized":"2024-09-22T03:12:34.047474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'818","reported_seq":870,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.194364+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:38.194364+0000","last_peered":"2024-09-22T03:12:38.194364+0000","last_clean":"2024-09-22T03:12:38.194364+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:38.194364+0000","last_undegraded":"2024-09-22T03:12:38.194364+0000","last_fullsized":"2024-09-22T03:12:38.194364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53673984,"num_objects":819,"num_object_clones":0,"num_object_copies":2457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":819,"num_write_kb":52416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.061211+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:34.061211+0000","last_peered":"2024-09-22T03:12:34.061211+0000","last_clean":"2024-09-22T03:12:34.061211+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:34.061211+0000","last_undegraded":"2024-09-22T03:12:34.061211+0000","last_fullsized":"2024-09-22T03:12:34.061211+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'839","reported_seq":1020,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.205415+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:38.205415+0000","last_peered":"2024-09-22T03:12:38.205415+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:38.205415+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55050240,"num_objects":840,"num_object_clones":0,"num_object_copies":2520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":53760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4587520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":414}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.056505+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:34.056505+0000","last_peered":"2024-09-22T03:12:34.056505+0000","last_clean":"2024-09-22T03:12:34.056505+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:34.056505+0000","last_undegraded":"2024-09-22T03:12:34.056505+0000","last_fullsized":"2024-09-22T03:12:34.056505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":869728256,"num_objects":13271,"num_object_clones":0,"num_object_copies":39813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13271,"num_write_kb":849344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":18808832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127369216,"data_stored":2037907456,"data_compressed":96767420,"data_compressed_allocated":127369216,"data_compressed_original":2037907456,"omap_allocated":0,"internal_metadata":0},"log_size":13255,"ondisk_log_size":13255,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99396,"kb_used_data":46196,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94272444,"statfs":{"total":96636764160,"available":96534982656,"internally_reserved":0,"allocated":47304704,"data_stored":752899596,"data_compressed":35746582,"data_compressed_allocated":47050752,"data_compressed_original":752812032,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63180,"kb_used_data":36348,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94308660,"statfs":{"total":96636764160,"available":96572067840,"internally_reserved":0,"allocated":37220352,"data_stored":591157292,"data_compressed":28068355,"data_compressed_allocated":36945920,"data_compressed_original":591052800,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149701,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54104,"kb_used_data":14152,"kb_used_omap":0,"kb_used_meta":39936,"kb_avail":94317736,"statfs":{"total":96636764160,"available":96581361664,"internally_reserved":0,"allocated":14491648,"data_stored":227498028,"data_compressed":10799753,"data_compressed_allocated":14217216,"data_compressed_original":227393536,"omap_allocated":0,"internal_metadata":40894464},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62124,"kb_used_data":35292,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94309716,"statfs":{"total":96636764160,"available":96573149184,"internally_reserved":0,"allocated":36139008,"data_stored":573921305,"data_compressed":27249854,"data_compressed_allocated":35868672,"data_compressed_original":573816832,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36638720,"data_stored":586219520,"data_compressed":27836058,"data_compressed_allocated":36638720,"data_compressed_original":586219520,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37736448,"data_stored":603783168,"data_compressed":28670119,"data_compressed_allocated":37736448,"data_compressed_original":603783168,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48115712,"data_stored":769851392,"data_compressed":36555702,"data_compressed_allocated":48115712,"data_compressed_original":769851392,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:39.672 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:12:39.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:39.828+0000 7f7b2aed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:39.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:39.838+0000 7f7b2aed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:40.284 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:40.285 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5687/39825 objects degraded (14.280%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":105,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":30,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":13275,"data_bytes":870318643,"bytes_used":285495296,"bytes_avail":386261561344,"bytes_total":386547056640,"degraded_objects":5687,"degraded_total":39825,"degraded_ratio":0.14279974890144381,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":2602141,"recovering_keys_per_sec":0,"num_objects_recovered":287,"num_bytes_recovered":18808832,"num_keys_recovered":0,"write_bytes_sec":21805398,"read_op_per_sec":0,"write_op_per_sec":332},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:12:40.304 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:12:40.304 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:40.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:40.450+0000 7fa8a0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:40.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:40.461+0000 7fa8a0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:40.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 85 16 14070 14054 10.3327 19.375 0.0346631 0.0967546 2024-09-22T03:12:40.773 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:40.802 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:40.804 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:40.829 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-22T03:12:39.242226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":870318643,"num_objects":13275,"num_object_clones":0,"num_object_copies":39825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13410,"num_write_kb":851240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":18808832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13355,"ondisk_log_size":13355,"up":75,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":278804,"kb_used_data":131988,"kb_used_omap":0,"kb_used_meta":146752,"kb_avail":377208556,"statfs":{"total":386547056640,"available":386261561344,"internally_reserved":0,"allocated":135155712,"data_stored":2145476221,"data_compressed":101864544,"data_compressed_allocated":134082560,"data_compressed_original":2145075200,"omap_allocated":0,"internal_metadata":150274048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":157614080,"num_objects":2405,"num_object_clones":0,"num_object_copies":7215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2405,"num_write_kb":153920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":18808832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.228213"},"pg_stats":[{"pgid":"3.f","version":"30'859","reported_seq":903,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:37.212616+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:37.212616+0000","last_peered":"2024-09-22T03:12:37.212616+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:37.212616+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56360960,"num_objects":860,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":860,"num_write_kb":55040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":325}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"30'853","reported_seq":899,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.005624+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:38.005624+0000","last_peered":"2024-09-22T03:12:38.005624+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:38.005624+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55967744,"num_objects":854,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":854,"num_write_kb":54656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":319}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'806","reported_seq":847,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:36.905037+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:36.905037+0000","last_peered":"2024-09-22T03:12:36.905037+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:36.905037+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52887552,"num_objects":807,"num_object_clones":0,"num_object_copies":2421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":807,"num_write_kb":51648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"30'849","reported_seq":901,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:37.189698+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:37.189698+0000","last_peered":"2024-09-22T03:12:37.189698+0000","last_clean":"2024-09-22T03:12:37.189698+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:37.189698+0000","last_undegraded":"2024-09-22T03:12:37.189698+0000","last_fullsized":"2024-09-22T03:12:37.189698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55705600,"num_objects":850,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":850,"num_write_kb":54400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'836","reported_seq":888,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.230219+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:38.230219+0000","last_peered":"2024-09-22T03:12:38.230219+0000","last_clean":"2024-09-22T03:12:38.230219+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:38.230219+0000","last_undegraded":"2024-09-22T03:12:38.230219+0000","last_fullsized":"2024-09-22T03:12:38.230219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":837,"num_write_kb":53568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"30'800","reported_seq":1132,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:37.236297+0000","last_change":"2024-09-22T03:12:32.631272+0000","last_active":"2024-09-22T03:12:37.236297+0000","last_peered":"2024-09-22T03:12:37.236297+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:32.536732+0000","last_became_peered":"2024-09-22T03:12:32.536732+0000","last_unstale":"2024-09-22T03:12:37.236297+0000","last_undegraded":"2024-09-22T03:12:32.106233+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52494336,"num_objects":801,"num_object_clones":0,"num_object_copies":2403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":801,"num_write_kb":51264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":9371648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"30'800","reported_seq":851,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:37.099249+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:37.099249+0000","last_peered":"2024-09-22T03:12:37.099249+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:37.099249+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52494336,"num_objects":801,"num_object_clones":0,"num_object_copies":2403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":801,"num_write_kb":51264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'862","reported_seq":1033,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.184437+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:38.184437+0000","last_peered":"2024-09-22T03:12:38.184437+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:38.184437+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":863,"num_write_kb":55232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":423}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047905+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:34.047905+0000","last_peered":"2024-09-22T03:12:34.047905+0000","last_clean":"2024-09-22T03:12:34.047905+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:34.047905+0000","last_undegraded":"2024-09-22T03:12:34.047905+0000","last_fullsized":"2024-09-22T03:12:34.047905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"30'841","reported_seq":880,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:36.877609+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:36.877609+0000","last_peered":"2024-09-22T03:12:36.877609+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:36.877609+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55181312,"num_objects":842,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":842,"num_write_kb":53888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.060241+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:34.060241+0000","last_peered":"2024-09-22T03:12:34.060241+0000","last_clean":"2024-09-22T03:12:34.060241+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:34.060241+0000","last_undegraded":"2024-09-22T03:12:34.060241+0000","last_fullsized":"2024-09-22T03:12:34.060241+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'823","reported_seq":875,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.227947+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:38.227947+0000","last_peered":"2024-09-22T03:12:38.227947+0000","last_clean":"2024-09-22T03:12:38.227947+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:38.227947+0000","last_undegraded":"2024-09-22T03:12:38.227947+0000","last_fullsized":"2024-09-22T03:12:38.227947+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54001664,"num_objects":824,"num_object_clones":0,"num_object_copies":2472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":52736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":73,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.043071+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:34.043071+0000","last_peered":"2024-09-22T03:12:34.043071+0000","last_clean":"2024-09-22T03:12:34.043071+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:34.043071+0000","last_undegraded":"2024-09-22T03:12:34.043071+0000","last_fullsized":"2024-09-22T03:12:34.043071+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"30'835","reported_seq":884,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.166335+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:38.166335+0000","last_peered":"2024-09-22T03:12:38.166335+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:38.166335+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54788096,"num_objects":836,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":836,"num_write_kb":53504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":344}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":72,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.072611+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:34.072611+0000","last_peered":"2024-09-22T03:12:34.072611+0000","last_clean":"2024-09-22T03:12:34.072611+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:34.072611+0000","last_undegraded":"2024-09-22T03:12:34.072611+0000","last_fullsized":"2024-09-22T03:12:34.072611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":195,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.067653+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:34.067653+0000","last_peered":"2024-09-22T03:12:34.067653+0000","last_clean":"2024-09-22T03:12:34.067653+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:34.067653+0000","last_undegraded":"2024-09-22T03:12:34.067653+0000","last_fullsized":"2024-09-22T03:12:34.067653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'797","reported_seq":841,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.145900+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:38.145900+0000","last_peered":"2024-09-22T03:12:38.145900+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:38.145900+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52297728,"num_objects":798,"num_object_clones":0,"num_object_copies":2394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":798,"num_write_kb":51072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":326}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"30'830","reported_seq":881,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.169458+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:38.169458+0000","last_peered":"2024-09-22T03:12:38.169458+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:38.169458+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54460416,"num_objects":831,"num_object_clones":0,"num_object_copies":2493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":53184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":330}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.055738+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:34.055738+0000","last_peered":"2024-09-22T03:12:34.055738+0000","last_clean":"2024-09-22T03:12:34.055738+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:34.055738+0000","last_undegraded":"2024-09-22T03:12:34.055738+0000","last_fullsized":"2024-09-22T03:12:34.055738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'807","reported_seq":856,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.154929+0000","last_change":"2024-09-22T03:12:32.629406+0000","last_active":"2024-09-22T03:12:38.154929+0000","last_peered":"2024-09-22T03:12:38.154929+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:38.154929+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52953088,"num_objects":808,"num_object_clones":0,"num_object_copies":2424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":51712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":328}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047474+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:34.047474+0000","last_peered":"2024-09-22T03:12:34.047474+0000","last_clean":"2024-09-22T03:12:34.047474+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:34.047474+0000","last_undegraded":"2024-09-22T03:12:34.047474+0000","last_fullsized":"2024-09-22T03:12:34.047474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'818","reported_seq":870,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.194364+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:38.194364+0000","last_peered":"2024-09-22T03:12:38.194364+0000","last_clean":"2024-09-22T03:12:38.194364+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:38.194364+0000","last_undegraded":"2024-09-22T03:12:38.194364+0000","last_fullsized":"2024-09-22T03:12:38.194364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53673984,"num_objects":819,"num_object_clones":0,"num_object_copies":2457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":819,"num_write_kb":52416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.061211+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:34.061211+0000","last_peered":"2024-09-22T03:12:34.061211+0000","last_clean":"2024-09-22T03:12:34.061211+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:34.061211+0000","last_undegraded":"2024-09-22T03:12:34.061211+0000","last_fullsized":"2024-09-22T03:12:34.061211+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'839","reported_seq":1020,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.205415+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:38.205415+0000","last_peered":"2024-09-22T03:12:38.205415+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:38.205415+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55050240,"num_objects":840,"num_object_clones":0,"num_object_copies":2520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":53760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4587520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":414}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.056505+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:34.056505+0000","last_peered":"2024-09-22T03:12:34.056505+0000","last_clean":"2024-09-22T03:12:34.056505+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:34.056505+0000","last_undegraded":"2024-09-22T03:12:34.056505+0000","last_fullsized":"2024-09-22T03:12:34.056505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":869728256,"num_objects":13271,"num_object_clones":0,"num_object_copies":39813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13271,"num_write_kb":849344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":18808832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127369216,"data_stored":2037907456,"data_compressed":96767420,"data_compressed_allocated":127369216,"data_compressed_original":2037907456,"omap_allocated":0,"internal_metadata":0},"log_size":13255,"ondisk_log_size":13255,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99396,"kb_used_data":46196,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94272444,"statfs":{"total":96636764160,"available":96534982656,"internally_reserved":0,"allocated":47304704,"data_stored":752899596,"data_compressed":35746582,"data_compressed_allocated":47050752,"data_compressed_original":752812032,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63180,"kb_used_data":36348,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94308660,"statfs":{"total":96636764160,"available":96572067840,"internally_reserved":0,"allocated":37220352,"data_stored":591157292,"data_compressed":28068355,"data_compressed_allocated":36945920,"data_compressed_original":591052800,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149701,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54104,"kb_used_data":14152,"kb_used_omap":0,"kb_used_meta":39936,"kb_avail":94317736,"statfs":{"total":96636764160,"available":96581361664,"internally_reserved":0,"allocated":14491648,"data_stored":227498028,"data_compressed":10799753,"data_compressed_allocated":14217216,"data_compressed_original":227393536,"omap_allocated":0,"internal_metadata":40894464},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62124,"kb_used_data":35292,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94309716,"statfs":{"total":96636764160,"available":96573149184,"internally_reserved":0,"allocated":36139008,"data_stored":573921305,"data_compressed":27249854,"data_compressed_allocated":35868672,"data_compressed_original":573816832,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36638720,"data_stored":586219520,"data_compressed":27836058,"data_compressed_allocated":36638720,"data_compressed_original":586219520,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37736448,"data_stored":603783168,"data_compressed":28670119,"data_compressed_allocated":37736448,"data_compressed_original":603783168,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48115712,"data_stored":769851392,"data_compressed":36555702,"data_compressed_allocated":48115712,"data_compressed_original":769851392,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:40.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:40.926+0000 7fc8de957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:40.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:40.935+0000 7fc8de957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:41.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 86 16 14357 14341 10.4211 17.9375 0.0363148 0.0959207 2024-09-22T03:12:42.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 87 16 14667 14651 10.524 19.375 0.0285106 0.0949887 2024-09-22T03:12:43.108 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:43.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:43.262+0000 7fa287557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:43.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:43.270+0000 7fa287557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:43.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 88 16 14978 14962 10.6252 19.4375 0.0919889 0.0940759 2024-09-22T03:12:43.831 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:44.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:43.996+0000 7f9b75e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:44.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:44.010+0000 7f9b75e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:44.340 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:44.340 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:44.360 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-22T03:12:43.243087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":907346483,"num_objects":13840,"num_object_clones":0,"num_object_copies":41520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13975,"num_write_kb":887400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":29949952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13921,"ondisk_log_size":13921,"up":75,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":280384,"kb_used_data":141888,"kb_used_omap":0,"kb_used_meta":138432,"kb_avail":377206976,"statfs":{"total":386547056640,"available":386259943424,"internally_reserved":0,"allocated":145293312,"data_stored":2307677821,"data_compressed":109566744,"data_compressed_allocated":144220160,"data_compressed_original":2307276800,"omap_allocated":0,"internal_metadata":141754368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":169213952,"num_objects":2582,"num_object_clones":0,"num_object_copies":7746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2582,"num_write_kb":165248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":29949952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002177"},"pg_stats":[{"pgid":"3.f","version":"30'953","reported_seq":997,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.128811+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:42.128811+0000","last_peered":"2024-09-22T03:12:42.128811+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:42.128811+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62521344,"num_objects":954,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":954,"num_write_kb":61056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"30'853","reported_seq":899,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.005624+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:38.005624+0000","last_peered":"2024-09-22T03:12:38.005624+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:38.005624+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55967744,"num_objects":854,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":854,"num_write_kb":54656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":319}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'909","reported_seq":950,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.168837+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:42.168837+0000","last_peered":"2024-09-22T03:12:42.168837+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:42.168837+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59637760,"num_objects":910,"num_object_clones":0,"num_object_copies":2730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":910,"num_write_kb":58240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"30'940","reported_seq":992,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:42.154367+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:42.154367+0000","last_peered":"2024-09-22T03:12:42.154367+0000","last_clean":"2024-09-22T03:12:42.154367+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:42.154367+0000","last_undegraded":"2024-09-22T03:12:42.154367+0000","last_fullsized":"2024-09-22T03:12:42.154367+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":941,"num_write_kb":60224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'836","reported_seq":888,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.230219+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:38.230219+0000","last_peered":"2024-09-22T03:12:38.230219+0000","last_clean":"2024-09-22T03:12:38.230219+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:38.230219+0000","last_undegraded":"2024-09-22T03:12:38.230219+0000","last_fullsized":"2024-09-22T03:12:38.230219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":837,"num_write_kb":53568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"30'892","reported_seq":1563,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.237194+0000","last_change":"2024-09-22T03:12:32.631272+0000","last_active":"2024-09-22T03:12:42.237194+0000","last_peered":"2024-09-22T03:12:42.237194+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:32.536732+0000","last_became_peered":"2024-09-22T03:12:32.536732+0000","last_unstale":"2024-09-22T03:12:42.237194+0000","last_undegraded":"2024-09-22T03:12:32.106233+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58458112,"num_objects":892,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":892,"num_write_kb":57088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":20512768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"30'892","reported_seq":943,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.235115+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:42.235115+0000","last_peered":"2024-09-22T03:12:42.235115+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:42.235115+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'862","reported_seq":1033,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.184437+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:38.184437+0000","last_peered":"2024-09-22T03:12:38.184437+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:38.184437+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":863,"num_write_kb":55232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":423}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047905+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:34.047905+0000","last_peered":"2024-09-22T03:12:34.047905+0000","last_clean":"2024-09-22T03:12:34.047905+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:34.047905+0000","last_undegraded":"2024-09-22T03:12:34.047905+0000","last_fullsized":"2024-09-22T03:12:34.047905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"30'935","reported_seq":974,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.083400+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:42.083400+0000","last_peered":"2024-09-22T03:12:42.083400+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:42.083400+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61341696,"num_objects":936,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":59904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.060241+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:34.060241+0000","last_peered":"2024-09-22T03:12:34.060241+0000","last_clean":"2024-09-22T03:12:34.060241+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:34.060241+0000","last_undegraded":"2024-09-22T03:12:34.060241+0000","last_fullsized":"2024-09-22T03:12:34.060241+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'823","reported_seq":875,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.227947+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:38.227947+0000","last_peered":"2024-09-22T03:12:38.227947+0000","last_clean":"2024-09-22T03:12:38.227947+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:38.227947+0000","last_undegraded":"2024-09-22T03:12:38.227947+0000","last_fullsized":"2024-09-22T03:12:38.227947+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54001664,"num_objects":824,"num_object_clones":0,"num_object_copies":2472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":52736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":73,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.043071+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:34.043071+0000","last_peered":"2024-09-22T03:12:34.043071+0000","last_clean":"2024-09-22T03:12:34.043071+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:34.043071+0000","last_undegraded":"2024-09-22T03:12:34.043071+0000","last_fullsized":"2024-09-22T03:12:34.043071+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"30'835","reported_seq":884,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.166335+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:38.166335+0000","last_peered":"2024-09-22T03:12:38.166335+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:38.166335+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54788096,"num_objects":836,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":836,"num_write_kb":53504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":344}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":72,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.072611+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:34.072611+0000","last_peered":"2024-09-22T03:12:34.072611+0000","last_clean":"2024-09-22T03:12:34.072611+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:34.072611+0000","last_undegraded":"2024-09-22T03:12:34.072611+0000","last_fullsized":"2024-09-22T03:12:34.072611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":195,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.067653+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:34.067653+0000","last_peered":"2024-09-22T03:12:34.067653+0000","last_clean":"2024-09-22T03:12:34.067653+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:34.067653+0000","last_undegraded":"2024-09-22T03:12:34.067653+0000","last_fullsized":"2024-09-22T03:12:34.067653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'797","reported_seq":841,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.145900+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:38.145900+0000","last_peered":"2024-09-22T03:12:38.145900+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:38.145900+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52297728,"num_objects":798,"num_object_clones":0,"num_object_copies":2394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":798,"num_write_kb":51072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":326}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"30'830","reported_seq":881,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.169458+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:38.169458+0000","last_peered":"2024-09-22T03:12:38.169458+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:38.169458+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54460416,"num_objects":831,"num_object_clones":0,"num_object_copies":2493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":53184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":330}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.055738+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:34.055738+0000","last_peered":"2024-09-22T03:12:34.055738+0000","last_clean":"2024-09-22T03:12:34.055738+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:34.055738+0000","last_undegraded":"2024-09-22T03:12:34.055738+0000","last_fullsized":"2024-09-22T03:12:34.055738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'807","reported_seq":856,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.154929+0000","last_change":"2024-09-22T03:12:32.629406+0000","last_active":"2024-09-22T03:12:38.154929+0000","last_peered":"2024-09-22T03:12:38.154929+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:38.154929+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52953088,"num_objects":808,"num_object_clones":0,"num_object_copies":2424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":51712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":328}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047474+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:34.047474+0000","last_peered":"2024-09-22T03:12:34.047474+0000","last_clean":"2024-09-22T03:12:34.047474+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:34.047474+0000","last_undegraded":"2024-09-22T03:12:34.047474+0000","last_fullsized":"2024-09-22T03:12:34.047474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'818","reported_seq":870,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:38.194364+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:38.194364+0000","last_peered":"2024-09-22T03:12:38.194364+0000","last_clean":"2024-09-22T03:12:38.194364+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:38.194364+0000","last_undegraded":"2024-09-22T03:12:38.194364+0000","last_fullsized":"2024-09-22T03:12:38.194364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53673984,"num_objects":819,"num_object_clones":0,"num_object_copies":2457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":819,"num_write_kb":52416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.061211+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:34.061211+0000","last_peered":"2024-09-22T03:12:34.061211+0000","last_clean":"2024-09-22T03:12:34.061211+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:34.061211+0000","last_undegraded":"2024-09-22T03:12:34.061211+0000","last_fullsized":"2024-09-22T03:12:34.061211+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'839","reported_seq":1020,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:38.205415+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:38.205415+0000","last_peered":"2024-09-22T03:12:38.205415+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:38.205415+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55050240,"num_objects":840,"num_object_clones":0,"num_object_copies":2520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":53760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4587520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":414}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.056505+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:34.056505+0000","last_peered":"2024-09-22T03:12:34.056505+0000","last_clean":"2024-09-22T03:12:34.056505+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:34.056505+0000","last_undegraded":"2024-09-22T03:12:34.056505+0000","last_fullsized":"2024-09-22T03:12:34.056505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":906756096,"num_objects":13836,"num_object_clones":0,"num_object_copies":41508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13836,"num_write_kb":885504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":29949952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131637248,"data_stored":2106195968,"data_compressed":100010124,"data_compressed_allocated":131637248,"data_compressed_original":2106195968,"omap_allocated":0,"internal_metadata":0},"log_size":13821,"ondisk_log_size":13821,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99396,"kb_used_data":46196,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94272444,"statfs":{"total":96636764160,"available":96534982656,"internally_reserved":0,"allocated":47304704,"data_stored":752899596,"data_compressed":35746582,"data_compressed_allocated":47050752,"data_compressed_original":752812032,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63180,"kb_used_data":36348,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94308660,"statfs":{"total":96636764160,"available":96572067840,"internally_reserved":0,"allocated":37220352,"data_stored":591157292,"data_compressed":28068355,"data_compressed_allocated":36945920,"data_compressed_original":591052800,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149706,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47352,"kb_used_data":19880,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":94324488,"statfs":{"total":96636764160,"available":96588275712,"internally_reserved":0,"allocated":20357120,"data_stored":321345580,"data_compressed":15256137,"data_compressed_allocated":20082688,"data_compressed_original":321241088,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70456,"kb_used_data":39464,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94301384,"statfs":{"total":96636764160,"available":96564617216,"internally_reserved":0,"allocated":40411136,"data_stored":642275353,"data_compressed":30495670,"data_compressed_allocated":40140800,"data_compressed_original":642170880,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40906752,"data_stored":654508032,"data_compressed":31078762,"data_compressed_allocated":40906752,"data_compressed_original":654508032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37736448,"data_stored":603783168,"data_compressed":28670119,"data_compressed_allocated":37736448,"data_compressed_original":603783168,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48115712,"data_stored":769851392,"data_compressed":36555702,"data_compressed_allocated":48115712,"data_compressed_original":769851392,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:44.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:12:44.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 89 16 15276 15260 10.7151 18.625 0.098067 0.0932891 2024-09-22T03:12:44.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:44.516+0000 7fbd66418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:44.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:44.525+0000 7fbd66418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:44.956 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:44.956 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5517/41520 objects degraded (13.288%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":109,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":30,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":13840,"data_bytes":907346483,"bytes_used":287113216,"bytes_avail":386259943424,"bytes_total":386547056640,"degraded_objects":5517,"degraded_total":41520,"degraded_ratio":0.13287572254335261,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":2994343,"recovering_keys_per_sec":0,"num_objects_recovered":457,"num_bytes_recovered":29949952,"num_keys_recovered":0,"write_bytes_sec":16917710,"read_op_per_sec":0,"write_op_per_sec":258},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:12:44.980 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:12:44.980 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:45.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:45.140+0000 7f318ba2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:45.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:45.149+0000 7f318ba2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:45.463 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:45.463 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:45.481 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-22T03:12:45.243512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":969540147,"num_objects":14789,"num_object_clones":0,"num_object_copies":44367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":14924,"num_write_kb":948136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":583,"num_bytes_recovered":38207488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14872,"ondisk_log_size":14872,"up":75,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":298128,"kb_used_data":151308,"kb_used_omap":0,"kb_used_meta":146752,"kb_avail":377189232,"statfs":{"total":386547056640,"available":386241773568,"internally_reserved":0,"allocated":154939392,"data_stored":2462015101,"data_compressed":116895504,"data_compressed_allocated":153866240,"data_compressed_original":2461614080,"omap_allocated":0,"internal_metadata":150274048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":195559424,"num_objects":2984,"num_object_clones":0,"num_object_copies":8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2984,"num_write_kb":190976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":37027840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.217556"},"pg_stats":[{"pgid":"3.f","version":"30'953","reported_seq":997,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.128811+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:42.128811+0000","last_peered":"2024-09-22T03:12:42.128811+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:42.128811+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62521344,"num_objects":954,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":954,"num_write_kb":61056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"30'951","reported_seq":997,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:43.127269+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:43.127269+0000","last_peered":"2024-09-22T03:12:43.127269+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:43.127269+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62390272,"num_objects":952,"num_object_clones":0,"num_object_copies":2856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":60928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":417}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'909","reported_seq":950,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.168837+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:42.168837+0000","last_peered":"2024-09-22T03:12:42.168837+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:42.168837+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59637760,"num_objects":910,"num_object_clones":0,"num_object_copies":2730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":910,"num_write_kb":58240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"30'940","reported_seq":992,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:42.154367+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:42.154367+0000","last_peered":"2024-09-22T03:12:42.154367+0000","last_clean":"2024-09-22T03:12:42.154367+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:42.154367+0000","last_undegraded":"2024-09-22T03:12:42.154367+0000","last_fullsized":"2024-09-22T03:12:42.154367+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":941,"num_write_kb":60224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'927","reported_seq":979,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:43.112040+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:43.112040+0000","last_peered":"2024-09-22T03:12:43.112040+0000","last_clean":"2024-09-22T03:12:43.112040+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:43.112040+0000","last_undegraded":"2024-09-22T03:12:43.112040+0000","last_fullsized":"2024-09-22T03:12:43.112040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60817408,"num_objects":928,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":59392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"30'892","reported_seq":1563,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.237194+0000","last_change":"2024-09-22T03:12:32.631272+0000","last_active":"2024-09-22T03:12:42.237194+0000","last_peered":"2024-09-22T03:12:42.237194+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:32.536732+0000","last_became_peered":"2024-09-22T03:12:32.536732+0000","last_unstale":"2024-09-22T03:12:42.237194+0000","last_undegraded":"2024-09-22T03:12:32.106233+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58458112,"num_objects":892,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":892,"num_write_kb":57088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":20512768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"30'892","reported_seq":943,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.235115+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:42.235115+0000","last_peered":"2024-09-22T03:12:42.235115+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:42.235115+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'971","reported_seq":1264,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:43.218476+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:43.218476+0000","last_peered":"2024-09-22T03:12:43.218476+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:43.218476+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63635456,"num_objects":971,"num_object_clones":0,"num_object_copies":2913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":971,"num_write_kb":62144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":8257536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":592}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047905+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:34.047905+0000","last_peered":"2024-09-22T03:12:34.047905+0000","last_clean":"2024-09-22T03:12:34.047905+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:34.047905+0000","last_undegraded":"2024-09-22T03:12:34.047905+0000","last_fullsized":"2024-09-22T03:12:34.047905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"30'935","reported_seq":974,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.083400+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:42.083400+0000","last_peered":"2024-09-22T03:12:42.083400+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:42.083400+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61341696,"num_objects":936,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":59904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.060241+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:34.060241+0000","last_peered":"2024-09-22T03:12:34.060241+0000","last_clean":"2024-09-22T03:12:34.060241+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:34.060241+0000","last_undegraded":"2024-09-22T03:12:34.060241+0000","last_fullsized":"2024-09-22T03:12:34.060241+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'905","reported_seq":957,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:43.190191+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:43.190191+0000","last_peered":"2024-09-22T03:12:43.190191+0000","last_clean":"2024-09-22T03:12:43.190191+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:43.190191+0000","last_undegraded":"2024-09-22T03:12:43.190191+0000","last_fullsized":"2024-09-22T03:12:43.190191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59375616,"num_objects":906,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":906,"num_write_kb":57984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":73,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.043071+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:34.043071+0000","last_peered":"2024-09-22T03:12:34.043071+0000","last_clean":"2024-09-22T03:12:34.043071+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:34.043071+0000","last_undegraded":"2024-09-22T03:12:34.043071+0000","last_fullsized":"2024-09-22T03:12:34.043071+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"30'931","reported_seq":980,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:43.142314+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:43.142314+0000","last_peered":"2024-09-22T03:12:43.142314+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:43.142314+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61079552,"num_objects":932,"num_object_clones":0,"num_object_copies":2796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":932,"num_write_kb":59648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":440}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":72,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.072611+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:34.072611+0000","last_peered":"2024-09-22T03:12:34.072611+0000","last_clean":"2024-09-22T03:12:34.072611+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:34.072611+0000","last_undegraded":"2024-09-22T03:12:34.072611+0000","last_fullsized":"2024-09-22T03:12:34.072611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":195,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.067653+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:34.067653+0000","last_peered":"2024-09-22T03:12:34.067653+0000","last_clean":"2024-09-22T03:12:34.067653+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:34.067653+0000","last_undegraded":"2024-09-22T03:12:34.067653+0000","last_fullsized":"2024-09-22T03:12:34.067653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'911","reported_seq":955,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:43.233739+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:43.233739+0000","last_peered":"2024-09-22T03:12:43.233739+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:43.233739+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59768832,"num_objects":912,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":58368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":440}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"30'919","reported_seq":970,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:42.986106+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:42.986106+0000","last_peered":"2024-09-22T03:12:42.986106+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:42.986106+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60293120,"num_objects":920,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":920,"num_write_kb":58880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.055738+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:34.055738+0000","last_peered":"2024-09-22T03:12:34.055738+0000","last_clean":"2024-09-22T03:12:34.055738+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:34.055738+0000","last_undegraded":"2024-09-22T03:12:34.055738+0000","last_fullsized":"2024-09-22T03:12:34.055738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'898","reported_seq":947,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:43.188435+0000","last_change":"2024-09-22T03:12:32.629406+0000","last_active":"2024-09-22T03:12:43.188435+0000","last_peered":"2024-09-22T03:12:43.188435+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:43.188435+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":898,"log_dups_size":0,"ondisk_log_size":898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58916864,"num_objects":899,"num_object_clones":0,"num_object_copies":2697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":899,"num_write_kb":57536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":419}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.047474+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:34.047474+0000","last_peered":"2024-09-22T03:12:34.047474+0000","last_clean":"2024-09-22T03:12:34.047474+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:34.047474+0000","last_undegraded":"2024-09-22T03:12:34.047474+0000","last_fullsized":"2024-09-22T03:12:34.047474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'911","reported_seq":963,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:43.197668+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:43.197668+0000","last_peered":"2024-09-22T03:12:43.197668+0000","last_clean":"2024-09-22T03:12:43.197668+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:43.197668+0000","last_undegraded":"2024-09-22T03:12:43.197668+0000","last_fullsized":"2024-09-22T03:12:43.197668+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59768832,"num_objects":912,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":58368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.061211+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:34.061211+0000","last_peered":"2024-09-22T03:12:34.061211+0000","last_clean":"2024-09-22T03:12:34.061211+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:34.061211+0000","last_undegraded":"2024-09-22T03:12:34.061211+0000","last_fullsized":"2024-09-22T03:12:34.061211+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'927","reported_seq":1237,"reported_epoch":30,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:43.247731+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:43.247731+0000","last_peered":"2024-09-22T03:12:43.247731+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:43.247731+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60751872,"num_objects":927,"num_object_clones":0,"num_object_copies":2781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":927,"num_write_kb":59328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":8847360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.056505+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:34.056505+0000","last_peered":"2024-09-22T03:12:34.056505+0000","last_clean":"2024-09-22T03:12:34.056505+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:34.056505+0000","last_undegraded":"2024-09-22T03:12:34.056505+0000","last_fullsized":"2024-09-22T03:12:34.056505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":968949760,"num_objects":14785,"num_object_clones":0,"num_object_copies":44355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14785,"num_write_kb":946240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":583,"num_bytes_recovered":38207488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141262848,"data_stored":2260205568,"data_compressed":107323324,"data_compressed_allocated":141262848,"data_compressed_original":2260205568,"omap_allocated":0,"internal_metadata":0},"log_size":14772,"ondisk_log_size":14772,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108888,"kb_used_data":51528,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94262952,"statfs":{"total":96636764160,"available":96525262848,"internally_reserved":0,"allocated":52764672,"data_stored":840259084,"data_compressed":39894878,"data_compressed_allocated":52510720,"data_compressed_original":840171520,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71432,"kb_used_data":40436,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94300408,"statfs":{"total":96636764160,"available":96563617792,"internally_reserved":0,"allocated":41406464,"data_stored":658135084,"data_compressed":31248819,"data_compressed_allocated":41132032,"data_compressed_original":658030592,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149706,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47352,"kb_used_data":19880,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":94324488,"statfs":{"total":96636764160,"available":96588275712,"internally_reserved":0,"allocated":20357120,"data_stored":321345580,"data_compressed":15256137,"data_compressed_allocated":20082688,"data_compressed_original":321241088,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70456,"kb_used_data":39464,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94301384,"statfs":{"total":96636764160,"available":96564617216,"internally_reserved":0,"allocated":40411136,"data_stored":642275353,"data_compressed":30495670,"data_compressed_allocated":40140800,"data_compressed_original":642170880,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40906752,"data_stored":654508032,"data_compressed":31078762,"data_compressed_allocated":40906752,"data_compressed_original":654508032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41918464,"data_stored":670695424,"data_compressed":31847471,"data_compressed_allocated":41918464,"data_compressed_original":670695424,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53559296,"data_stored":856948736,"data_compressed":40691550,"data_compressed_allocated":53559296,"data_compressed_original":856948736,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:45.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 90 15 15575 15560 10.8044 18.75 0.0515059 0.0925145 2024-09-22T03:12:45.495 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:45.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:45.625+0000 7f6f1e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:45.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:45.631+0000 7f6f1e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:46.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total time run: 90.0485 2024-09-22T03:12:46.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total writes made: 15575 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Write size: 65536 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object size: 65536 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Bandwidth (MB/sec): 10.8101 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Bandwidth: 11.7448 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max bandwidth (MB/sec): 28.75 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min bandwidth (MB/sec): 0 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average IOPS: 172 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev IOPS: 187.925 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max IOPS: 460 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min IOPS: 0 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average Latency(s): 0.0924855 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Latency(s): 1.11627 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max latency(s): 31.8821 2024-09-22T03:12:46.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min latency(s): 0.00921296 2024-09-22T03:12:46.505 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:12:47.727 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:47.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:47.862+0000 7f48af26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:47.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:47.868+0000 7f48af26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:48.483 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:48.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:48.628+0000 7f81f20a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:48.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:48.641+0000 7f81f20a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:48.921 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:48.921 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:48.941 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-22T03:12:48.347647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":997196363,"num_objects":15212,"num_object_clones":0,"num_object_copies":45636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":16083,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16031,"ondisk_log_size":16031,"up":75,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":311832,"kb_used_data":165016,"kb_used_omap":0,"kb_used_meta":146752,"kb_avail":377175528,"statfs":{"total":386547056640,"available":386227740672,"internally_reserved":0,"allocated":168976384,"data_stored":2686541461,"data_compressed":127557784,"data_compressed_allocated":167899136,"data_compressed_original":2686140416,"omap_allocated":0,"internal_metadata":150274048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":126877720,"num_objects":1937,"num_object_clones":0,"num_object_copies":5811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2673,"num_write_kb":147457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":51707904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.105421"},"pg_stats":[{"pgid":"3.f","version":"30'1040","reported_seq":1084,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.098327+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:47.098327+0000","last_peered":"2024-09-22T03:12:47.098327+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:47.098327+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65732608,"num_objects":1003,"num_object_clones":0,"num_object_copies":3009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1041,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"31'1005","reported_seq":1053,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.725342+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:47.725342+0000","last_peered":"2024-09-22T03:12:47.725342+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:47.725342+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62783488,"num_objects":958,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'986","reported_seq":1028,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.135433+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:47.135433+0000","last_peered":"2024-09-22T03:12:47.135433+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:47.135433+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61472792,"num_objects":939,"num_object_clones":0,"num_object_copies":2817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":987,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"30'1013","reported_seq":1065,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.065588+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:47.065588+0000","last_peered":"2024-09-22T03:12:47.065588+0000","last_clean":"2024-09-22T03:12:47.065588+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:47.065588+0000","last_undegraded":"2024-09-22T03:12:47.065588+0000","last_fullsized":"2024-09-22T03:12:47.065588+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64487424,"num_objects":984,"num_object_clones":0,"num_object_copies":2952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1014,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"31'991","reported_seq":1046,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.695962+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:47.695962+0000","last_peered":"2024-09-22T03:12:47.695962+0000","last_clean":"2024-09-22T03:12:47.695962+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:47.695962+0000","last_undegraded":"2024-09-22T03:12:47.695962+0000","last_fullsized":"2024-09-22T03:12:47.695962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":960,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"30'977","reported_seq":2018,"reported_epoch":30,"state":"active+undersized+remapped","last_fresh":"2024-09-22T03:12:47.180992+0000","last_change":"2024-09-22T03:12:46.677602+0000","last_active":"2024-09-22T03:12:47.180992+0000","last_peered":"2024-09-22T03:12:47.180992+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:32.536732+0000","last_became_peered":"2024-09-22T03:12:32.536732+0000","last_unstale":"2024-09-22T03:12:47.180992+0000","last_undegraded":"2024-09-22T03:12:47.180992+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0,2","objects":950}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"30'983","reported_seq":1034,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.143043+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:47.143043+0000","last_peered":"2024-09-22T03:12:47.143043+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:47.143043+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61734912,"num_objects":942,"num_object_clones":0,"num_object_copies":2826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'1031","reported_seq":1585,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.248276+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:48.248276+0000","last_peered":"2024-09-22T03:12:48.248276+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:48.248276+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64684032,"num_objects":987,"num_object_clones":0,"num_object_copies":2961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1031,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":738}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.363044+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:47.363044+0000","last_peered":"2024-09-22T03:12:47.363044+0000","last_clean":"2024-09-22T03:12:47.363044+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:47.363044+0000","last_undegraded":"2024-09-22T03:12:47.363044+0000","last_fullsized":"2024-09-22T03:12:47.363044+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"30'1011","reported_seq":1050,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.124704+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:47.124704+0000","last_peered":"2024-09-22T03:12:47.124704+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:47.124704+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1011,"log_dups_size":0,"ondisk_log_size":1011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64094208,"num_objects":978,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1012,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.060241+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:34.060241+0000","last_peered":"2024-09-22T03:12:34.060241+0000","last_clean":"2024-09-22T03:12:34.060241+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:34.060241+0000","last_undegraded":"2024-09-22T03:12:34.060241+0000","last_fullsized":"2024-09-22T03:12:34.060241+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.830967+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:47.830967+0000","last_peered":"2024-09-22T03:12:47.830967+0000","last_clean":"2024-09-22T03:12:47.830967+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:47.830967+0000","last_undegraded":"2024-09-22T03:12:47.830967+0000","last_fullsized":"2024-09-22T03:12:47.830967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60686336,"num_objects":926,"num_object_clones":0,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":76,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.367365+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:47.367365+0000","last_peered":"2024-09-22T03:12:47.367365+0000","last_clean":"2024-09-22T03:12:47.367365+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:47.367365+0000","last_undegraded":"2024-09-22T03:12:47.367365+0000","last_fullsized":"2024-09-22T03:12:47.367365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"31'1005","reported_seq":1056,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.940948+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:47.940948+0000","last_peered":"2024-09-22T03:12:47.940948+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:47.940948+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.380545+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:47.380545+0000","last_peered":"2024-09-22T03:12:47.380545+0000","last_clean":"2024-09-22T03:12:47.380545+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:47.380545+0000","last_undegraded":"2024-09-22T03:12:47.380545+0000","last_fullsized":"2024-09-22T03:12:47.380545+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":195,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.067653+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:34.067653+0000","last_peered":"2024-09-22T03:12:34.067653+0000","last_clean":"2024-09-22T03:12:34.067653+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:34.067653+0000","last_undegraded":"2024-09-22T03:12:34.067653+0000","last_fullsized":"2024-09-22T03:12:34.067653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"31'989","reported_seq":1035,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.682510+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:47.682510+0000","last_peered":"2024-09-22T03:12:47.682510+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:47.682510+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60948480,"num_objects":930,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":990,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'977","reported_seq":1030,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.727320+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:47.727320+0000","last_peered":"2024-09-22T03:12:47.727320+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:47.727320+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61210624,"num_objects":934,"num_object_clones":0,"num_object_copies":2802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.055738+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:34.055738+0000","last_peered":"2024-09-22T03:12:34.055738+0000","last_clean":"2024-09-22T03:12:34.055738+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:34.055738+0000","last_undegraded":"2024-09-22T03:12:34.055738+0000","last_fullsized":"2024-09-22T03:12:34.055738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'966","reported_seq":1085,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.251580+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:12:48.251580+0000","last_peered":"2024-09-22T03:12:48.251580+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:48.251580+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59244544,"num_objects":904,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":966,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.368991+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:47.368991+0000","last_peered":"2024-09-22T03:12:47.368991+0000","last_clean":"2024-09-22T03:12:47.368991+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:47.368991+0000","last_undegraded":"2024-09-22T03:12:47.368991+0000","last_fullsized":"2024-09-22T03:12:47.368991+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.816973+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:47.816973+0000","last_peered":"2024-09-22T03:12:47.816973+0000","last_clean":"2024-09-22T03:12:47.816973+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:47.816973+0000","last_undegraded":"2024-09-22T03:12:47.816973+0000","last_fullsized":"2024-09-22T03:12:47.816973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.061211+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:34.061211+0000","last_peered":"2024-09-22T03:12:34.061211+0000","last_clean":"2024-09-22T03:12:34.061211+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:34.061211+0000","last_undegraded":"2024-09-22T03:12:34.061211+0000","last_fullsized":"2024-09-22T03:12:34.061211+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"31'1007","reported_seq":1609,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.246063+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:48.246063+0000","last_peered":"2024-09-22T03:12:48.246063+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:48.246063+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":18350080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":725}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.056505+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:34.056505+0000","last_peered":"2024-09-22T03:12:34.056505+0000","last_clean":"2024-09-22T03:12:34.056505+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:34.056505+0000","last_undegraded":"2024-09-22T03:12:34.056505+0000","last_fullsized":"2024-09-22T03:12:34.056505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":996605976,"num_objects":15208,"num_object_clones":0,"num_object_copies":45624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15944,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145002496,"data_stored":2319908912,"data_compressed":110159422,"data_compressed_allocated":144994304,"data_compressed_original":2319908864,"omap_allocated":0,"internal_metadata":0},"log_size":15931,"ondisk_log_size":15931,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98772,"kb_used_data":53892,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94273068,"statfs":{"total":96636764160,"available":96535621632,"internally_reserved":0,"allocated":55185408,"data_stored":878925348,"data_compressed":41731290,"data_compressed_allocated":54927360,"data_compressed_original":878837760,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77584,"kb_used_data":42432,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94294256,"statfs":{"total":96636764160,"available":96557318144,"internally_reserved":0,"allocated":43450368,"data_stored":690837548,"data_compressed":32801943,"data_compressed_allocated":43175936,"data_compressed_original":690733056,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149711,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57316,"kb_used_data":25684,"kb_used_omap":0,"kb_used_meta":31616,"kb_avail":94314524,"statfs":{"total":96636764160,"available":96578072576,"internally_reserved":0,"allocated":26300416,"data_stored":416438316,"data_compressed":19771649,"data_compressed_allocated":26025984,"data_compressed_original":416333824,"omap_allocated":0,"internal_metadata":32374784},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78160,"kb_used_data":43008,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94293680,"statfs":{"total":96636764160,"available":96556728320,"internally_reserved":0,"allocated":44040192,"data_stored":700340249,"data_compressed":33252902,"data_compressed_allocated":43769856,"data_compressed_original":700235776,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":42967040,"data_stored":687407128,"data_compressed":32641262,"data_compressed_allocated":42962944,"data_compressed_original":687407104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42733568,"data_stored":683737088,"data_compressed":32467093,"data_compressed_allocated":42733568,"data_compressed_original":683737088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":54423552,"data_stored":870711320,"data_compressed":41345526,"data_compressed_allocated":54419456,"data_compressed_original":870711296,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:48.942 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:12:49.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:49.090+0000 7fe8d0f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:49.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:49.098+0000 7fe8d0f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:49.490 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:49.490 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4898/45636 objects degraded (10.733%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":114,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":32,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":3},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":25,"num_pools":3,"num_objects":15212,"data_bytes":997196363,"bytes_used":319315968,"bytes_avail":386227740672,"bytes_total":386547056640,"degraded_objects":4898,"degraded_total":45636,"degraded_ratio":0.10732754842668069,"recovering_objects_per_sec":98,"recovering_bytes_per_sec":6462123,"recovering_keys_per_sec":0,"num_objects_recovered":789,"num_bytes_recovered":51707904,"num_keys_recovered":0,"read_bytes_sec":127,"write_bytes_sec":18870511,"read_op_per_sec":0,"write_op_per_sec":334},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:12:49.508 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:12:49.508 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:49.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:49.642+0000 7f2e404ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:49.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:49.650+0000 7f2e404ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:49.909 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:49.910 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:49.931 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-22T03:12:49.244339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":997196363,"num_objects":15212,"num_object_clones":0,"num_object_copies":45636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":16083,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16031,"ondisk_log_size":16031,"up":75,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":311832,"kb_used_data":165016,"kb_used_omap":0,"kb_used_meta":146752,"kb_avail":377175528,"statfs":{"total":386547056640,"available":386227740672,"internally_reserved":0,"allocated":168976384,"data_stored":2686541461,"data_compressed":127557784,"data_compressed_allocated":167899136,"data_compressed_original":2686140416,"omap_allocated":0,"internal_metadata":150274048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":126877720,"num_objects":1937,"num_object_clones":0,"num_object_copies":5811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2673,"num_write_kb":147457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":51707904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001689"},"pg_stats":[{"pgid":"3.f","version":"30'1040","reported_seq":1084,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.098327+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:47.098327+0000","last_peered":"2024-09-22T03:12:47.098327+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:47.098327+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65732608,"num_objects":1003,"num_object_clones":0,"num_object_copies":3009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1041,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"31'1005","reported_seq":1053,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.725342+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:47.725342+0000","last_peered":"2024-09-22T03:12:47.725342+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:47.725342+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62783488,"num_objects":958,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'986","reported_seq":1028,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.135433+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:47.135433+0000","last_peered":"2024-09-22T03:12:47.135433+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:47.135433+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61472792,"num_objects":939,"num_object_clones":0,"num_object_copies":2817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":987,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"30'1013","reported_seq":1065,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.065588+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:47.065588+0000","last_peered":"2024-09-22T03:12:47.065588+0000","last_clean":"2024-09-22T03:12:47.065588+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:47.065588+0000","last_undegraded":"2024-09-22T03:12:47.065588+0000","last_fullsized":"2024-09-22T03:12:47.065588+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64487424,"num_objects":984,"num_object_clones":0,"num_object_copies":2952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1014,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"31'991","reported_seq":1046,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.695962+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:47.695962+0000","last_peered":"2024-09-22T03:12:47.695962+0000","last_clean":"2024-09-22T03:12:47.695962+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:47.695962+0000","last_undegraded":"2024-09-22T03:12:47.695962+0000","last_fullsized":"2024-09-22T03:12:47.695962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":960,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"30'977","reported_seq":2018,"reported_epoch":30,"state":"active+undersized+remapped","last_fresh":"2024-09-22T03:12:47.180992+0000","last_change":"2024-09-22T03:12:46.677602+0000","last_active":"2024-09-22T03:12:47.180992+0000","last_peered":"2024-09-22T03:12:47.180992+0000","last_clean":"2024-09-22T03:11:38.282928+0000","last_became_active":"2024-09-22T03:12:32.536732+0000","last_became_peered":"2024-09-22T03:12:32.536732+0000","last_unstale":"2024-09-22T03:12:47.180992+0000","last_undegraded":"2024-09-22T03:12:47.180992+0000","last_fullsized":"2024-09-22T03:12:32.023911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0,2","objects":950}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"30'983","reported_seq":1034,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.143043+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:47.143043+0000","last_peered":"2024-09-22T03:12:47.143043+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:47.143043+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61734912,"num_objects":942,"num_object_clones":0,"num_object_copies":2826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'1031","reported_seq":1585,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.248276+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:48.248276+0000","last_peered":"2024-09-22T03:12:48.248276+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:48.248276+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64684032,"num_objects":987,"num_object_clones":0,"num_object_copies":2961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1031,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":738}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.363044+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:47.363044+0000","last_peered":"2024-09-22T03:12:47.363044+0000","last_clean":"2024-09-22T03:12:47.363044+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:47.363044+0000","last_undegraded":"2024-09-22T03:12:47.363044+0000","last_fullsized":"2024-09-22T03:12:47.363044+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"30'1011","reported_seq":1050,"reported_epoch":30,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.124704+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:47.124704+0000","last_peered":"2024-09-22T03:12:47.124704+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:47.124704+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1011,"log_dups_size":0,"ondisk_log_size":1011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64094208,"num_objects":978,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1012,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.060241+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:34.060241+0000","last_peered":"2024-09-22T03:12:34.060241+0000","last_clean":"2024-09-22T03:12:34.060241+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:34.060241+0000","last_undegraded":"2024-09-22T03:12:34.060241+0000","last_fullsized":"2024-09-22T03:12:34.060241+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.830967+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:47.830967+0000","last_peered":"2024-09-22T03:12:47.830967+0000","last_clean":"2024-09-22T03:12:47.830967+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:47.830967+0000","last_undegraded":"2024-09-22T03:12:47.830967+0000","last_fullsized":"2024-09-22T03:12:47.830967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60686336,"num_objects":926,"num_object_clones":0,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":76,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.367365+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:47.367365+0000","last_peered":"2024-09-22T03:12:47.367365+0000","last_clean":"2024-09-22T03:12:47.367365+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:47.367365+0000","last_undegraded":"2024-09-22T03:12:47.367365+0000","last_fullsized":"2024-09-22T03:12:47.367365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"31'1005","reported_seq":1056,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.940948+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:47.940948+0000","last_peered":"2024-09-22T03:12:47.940948+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:47.940948+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.380545+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:47.380545+0000","last_peered":"2024-09-22T03:12:47.380545+0000","last_clean":"2024-09-22T03:12:47.380545+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:47.380545+0000","last_undegraded":"2024-09-22T03:12:47.380545+0000","last_fullsized":"2024-09-22T03:12:47.380545+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":195,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.067653+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:34.067653+0000","last_peered":"2024-09-22T03:12:34.067653+0000","last_clean":"2024-09-22T03:12:34.067653+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:34.067653+0000","last_undegraded":"2024-09-22T03:12:34.067653+0000","last_fullsized":"2024-09-22T03:12:34.067653+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"31'989","reported_seq":1035,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.682510+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:47.682510+0000","last_peered":"2024-09-22T03:12:47.682510+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:47.682510+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60948480,"num_objects":930,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":990,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'977","reported_seq":1030,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.727320+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:47.727320+0000","last_peered":"2024-09-22T03:12:47.727320+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:47.727320+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61210624,"num_objects":934,"num_object_clones":0,"num_object_copies":2802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.055738+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:34.055738+0000","last_peered":"2024-09-22T03:12:34.055738+0000","last_clean":"2024-09-22T03:12:34.055738+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:34.055738+0000","last_undegraded":"2024-09-22T03:12:34.055738+0000","last_fullsized":"2024-09-22T03:12:34.055738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'966","reported_seq":1085,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.251580+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:12:48.251580+0000","last_peered":"2024-09-22T03:12:48.251580+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:48.251580+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59244544,"num_objects":904,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":966,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.368991+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:47.368991+0000","last_peered":"2024-09-22T03:12:47.368991+0000","last_clean":"2024-09-22T03:12:47.368991+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:47.368991+0000","last_undegraded":"2024-09-22T03:12:47.368991+0000","last_fullsized":"2024-09-22T03:12:47.368991+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.816973+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:47.816973+0000","last_peered":"2024-09-22T03:12:47.816973+0000","last_clean":"2024-09-22T03:12:47.816973+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:47.816973+0000","last_undegraded":"2024-09-22T03:12:47.816973+0000","last_fullsized":"2024-09-22T03:12:47.816973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.061211+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:34.061211+0000","last_peered":"2024-09-22T03:12:34.061211+0000","last_clean":"2024-09-22T03:12:34.061211+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:34.061211+0000","last_undegraded":"2024-09-22T03:12:34.061211+0000","last_fullsized":"2024-09-22T03:12:34.061211+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"31'1007","reported_seq":1609,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.246063+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:48.246063+0000","last_peered":"2024-09-22T03:12:48.246063+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:48.246063+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":18350080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":725}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T03:12:34.056505+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:34.056505+0000","last_peered":"2024-09-22T03:12:34.056505+0000","last_clean":"2024-09-22T03:12:34.056505+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:34.056505+0000","last_undegraded":"2024-09-22T03:12:34.056505+0000","last_fullsized":"2024-09-22T03:12:34.056505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":996605976,"num_objects":15208,"num_object_clones":0,"num_object_copies":45624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15944,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145002496,"data_stored":2319908912,"data_compressed":110159422,"data_compressed_allocated":144994304,"data_compressed_original":2319908864,"omap_allocated":0,"internal_metadata":0},"log_size":15931,"ondisk_log_size":15931,"up":48,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98772,"kb_used_data":53892,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94273068,"statfs":{"total":96636764160,"available":96535621632,"internally_reserved":0,"allocated":55185408,"data_stored":878925348,"data_compressed":41731290,"data_compressed_allocated":54927360,"data_compressed_original":878837760,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77584,"kb_used_data":42432,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94294256,"statfs":{"total":96636764160,"available":96557318144,"internally_reserved":0,"allocated":43450368,"data_stored":690837548,"data_compressed":32801943,"data_compressed_allocated":43175936,"data_compressed_original":690733056,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149711,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57316,"kb_used_data":25684,"kb_used_omap":0,"kb_used_meta":31616,"kb_avail":94314524,"statfs":{"total":96636764160,"available":96578072576,"internally_reserved":0,"allocated":26300416,"data_stored":416438316,"data_compressed":19771649,"data_compressed_allocated":26025984,"data_compressed_original":416333824,"omap_allocated":0,"internal_metadata":32374784},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78160,"kb_used_data":43008,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94293680,"statfs":{"total":96636764160,"available":96556728320,"internally_reserved":0,"allocated":44040192,"data_stored":700340249,"data_compressed":33252902,"data_compressed_allocated":43769856,"data_compressed_original":700235776,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":42967040,"data_stored":687407128,"data_compressed":32641262,"data_compressed_allocated":42962944,"data_compressed_original":687407104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42733568,"data_stored":683737088,"data_compressed":32467093,"data_compressed_allocated":42733568,"data_compressed_original":683737088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":54423552,"data_stored":870711320,"data_compressed":41345526,"data_compressed_allocated":54419456,"data_compressed_original":870711296,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:49.970 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:50.127 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:50.122+0000 7f97a3f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:50.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:50.132+0000 7f97a3f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:52.309 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:52.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:52.453+0000 7f32e9197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:52.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:52.461+0000 7f32e9197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:52.933 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:53.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:53.088+0000 7f7a6cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:53.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:53.096+0000 7f7a6cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:53.408 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:53.408 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:53.425 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-22T03:12:53.245177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":962134603,"num_objects":14677,"num_object_clones":0,"num_object_copies":44031,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":16618,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16566,"ondisk_log_size":16566,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":333828,"kb_used_data":160644,"kb_used_omap":0,"kb_used_meta":173120,"kb_avail":377153532,"statfs":{"total":386547056640,"available":386205216768,"internally_reserved":0,"allocated":164499456,"data_stored":2614269711,"data_compressed":124126818,"data_compressed_allocated":163381248,"data_compressed_original":2613854208,"omap_allocated":0,"internal_metadata":177274880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7405544,"num_objects":-112,"num_object_clones":0,"num_object_copies":-336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1694,"num_write_kb":50561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001665"},"pg_stats":[{"pgid":"3.f","version":"32'1121","reported_seq":1169,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186626+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:52.186626+0000","last_peered":"2024-09-22T03:12:52.186626+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:52.186626+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"31'1005","reported_seq":1053,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.725342+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:47.725342+0000","last_peered":"2024-09-22T03:12:47.725342+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:47.725342+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62783488,"num_objects":958,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1062","reported_seq":1108,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.164836+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:52.164836+0000","last_peered":"2024-09-22T03:12:52.164836+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:52.164836+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492056,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1063,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1103","reported_seq":1161,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.125973+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:52.125973+0000","last_peered":"2024-09-22T03:12:52.125973+0000","last_clean":"2024-09-22T03:12:52.125973+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:52.125973+0000","last_undegraded":"2024-09-22T03:12:52.125973+0000","last_fullsized":"2024-09-22T03:12:52.125973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1103,"log_dups_size":0,"ondisk_log_size":1103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58589184,"num_objects":894,"num_object_clones":0,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1104,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"31'991","reported_seq":1046,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.695962+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:47.695962+0000","last_peered":"2024-09-22T03:12:47.695962+0000","last_clean":"2024-09-22T03:12:47.695962+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:47.695962+0000","last_undegraded":"2024-09-22T03:12:47.695962+0000","last_fullsized":"2024-09-22T03:12:47.695962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":960,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1065","reported_seq":2115,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.098973+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:12:52.098973+0000","last_peered":"2024-09-22T03:12:52.098973+0000","last_clean":"2024-09-22T03:12:52.098973+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:12:52.098973+0000","last_undegraded":"2024-09-22T03:12:52.098973+0000","last_fullsized":"2024-09-22T03:12:52.098973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1065,"log_dups_size":0,"ondisk_log_size":1065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492032,"num_objects":862,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1066,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1080","reported_seq":1135,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.118209+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:52.118209+0000","last_peered":"2024-09-22T03:12:52.118209+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:52.118209+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55377920,"num_objects":845,"num_object_clones":0,"num_object_copies":2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1081,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'1031","reported_seq":1585,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.248276+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:48.248276+0000","last_peered":"2024-09-22T03:12:48.248276+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:48.248276+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64684032,"num_objects":987,"num_object_clones":0,"num_object_copies":2961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1031,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":738}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.363044+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:47.363044+0000","last_peered":"2024-09-22T03:12:47.363044+0000","last_clean":"2024-09-22T03:12:47.363044+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:47.363044+0000","last_undegraded":"2024-09-22T03:12:47.363044+0000","last_fullsized":"2024-09-22T03:12:47.363044+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1114","reported_seq":1157,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186620+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:52.186620+0000","last_peered":"2024-09-22T03:12:52.186620+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:52.186620+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57344000,"num_objects":875,"num_object_clones":0,"num_object_copies":2625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.830967+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:47.830967+0000","last_peered":"2024-09-22T03:12:47.830967+0000","last_clean":"2024-09-22T03:12:47.830967+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:47.830967+0000","last_undegraded":"2024-09-22T03:12:47.830967+0000","last_fullsized":"2024-09-22T03:12:47.830967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60686336,"num_objects":926,"num_object_clones":0,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":76,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.367365+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:47.367365+0000","last_peered":"2024-09-22T03:12:47.367365+0000","last_clean":"2024-09-22T03:12:47.367365+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:47.367365+0000","last_undegraded":"2024-09-22T03:12:47.367365+0000","last_fullsized":"2024-09-22T03:12:47.367365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"31'1005","reported_seq":1056,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.940948+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:47.940948+0000","last_peered":"2024-09-22T03:12:47.940948+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:47.940948+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.380545+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:47.380545+0000","last_peered":"2024-09-22T03:12:47.380545+0000","last_clean":"2024-09-22T03:12:47.380545+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:47.380545+0000","last_undegraded":"2024-09-22T03:12:47.380545+0000","last_fullsized":"2024-09-22T03:12:47.380545+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"31'989","reported_seq":1035,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.682510+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:47.682510+0000","last_peered":"2024-09-22T03:12:47.682510+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:47.682510+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60948480,"num_objects":930,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":990,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'977","reported_seq":1030,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.727320+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:47.727320+0000","last_peered":"2024-09-22T03:12:47.727320+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:47.727320+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61210624,"num_objects":934,"num_object_clones":0,"num_object_copies":2802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356684+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:48.356684+0000","last_peered":"2024-09-22T03:12:48.356684+0000","last_clean":"2024-09-22T03:12:48.356684+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:48.356684+0000","last_undegraded":"2024-09-22T03:12:48.356684+0000","last_fullsized":"2024-09-22T03:12:48.356684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'966","reported_seq":1085,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.251580+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:12:48.251580+0000","last_peered":"2024-09-22T03:12:48.251580+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:48.251580+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59244544,"num_objects":904,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":966,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.368991+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:47.368991+0000","last_peered":"2024-09-22T03:12:47.368991+0000","last_clean":"2024-09-22T03:12:47.368991+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:47.368991+0000","last_undegraded":"2024-09-22T03:12:47.368991+0000","last_fullsized":"2024-09-22T03:12:47.368991+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.816973+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:47.816973+0000","last_peered":"2024-09-22T03:12:47.816973+0000","last_clean":"2024-09-22T03:12:47.816973+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:47.816973+0000","last_undegraded":"2024-09-22T03:12:47.816973+0000","last_fullsized":"2024-09-22T03:12:47.816973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356390+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:48.356390+0000","last_peered":"2024-09-22T03:12:48.356390+0000","last_clean":"2024-09-22T03:12:48.356390+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:48.356390+0000","last_undegraded":"2024-09-22T03:12:48.356390+0000","last_fullsized":"2024-09-22T03:12:48.356390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"31'1007","reported_seq":1609,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.246063+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:48.246063+0000","last_peered":"2024-09-22T03:12:48.246063+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:48.246063+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":18350080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":725}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356172+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:48.356172+0000","last_peered":"2024-09-22T03:12:48.356172+0000","last_clean":"2024-09-22T03:12:48.356172+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:48.356172+0000","last_undegraded":"2024-09-22T03:12:48.356172+0000","last_fullsized":"2024-09-22T03:12:48.356172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":961544216,"num_objects":14673,"num_object_clones":0,"num_object_copies":44019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16479,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141041664,"data_stored":2256535600,"data_compressed":107150624,"data_compressed_allocated":141033472,"data_compressed_original":2256535552,"omap_allocated":0,"internal_metadata":0},"log_size":16466,"ondisk_log_size":16466,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98772,"kb_used_data":53892,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94273068,"statfs":{"total":96636764160,"available":96535621632,"internally_reserved":0,"allocated":55185408,"data_stored":878925348,"data_compressed":41731290,"data_compressed_allocated":54927360,"data_compressed_original":878837760,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77584,"kb_used_data":42432,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94294256,"statfs":{"total":96636764160,"available":96557318144,"internally_reserved":0,"allocated":43450368,"data_stored":690837548,"data_compressed":32801943,"data_compressed_allocated":43175936,"data_compressed_original":690733056,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149716,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64964,"kb_used_data":25012,"kb_used_omap":0,"kb_used_meta":39936,"kb_avail":94306876,"statfs":{"total":96636764160,"available":96570241024,"internally_reserved":0,"allocated":25612288,"data_stored":405107817,"data_compressed":19234061,"data_compressed_allocated":25317376,"data_compressed_original":404996096,"omap_allocated":0,"internal_metadata":40894464},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92508,"kb_used_data":39308,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94279332,"statfs":{"total":96636764160,"available":96542035968,"internally_reserved":0,"allocated":40251392,"data_stored":639398998,"data_compressed":30359524,"data_compressed_allocated":39960576,"data_compressed_original":639287296,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39006208,"data_stored":624033816,"data_compressed":29632464,"data_compressed_allocated":39002112,"data_compressed_original":624033792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42733568,"data_stored":683737088,"data_compressed":32467093,"data_compressed_allocated":42733568,"data_compressed_original":683737088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":54423552,"data_stored":870711320,"data_compressed":41345526,"data_compressed_allocated":54419456,"data_compressed_original":870711296,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:53.426 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:12:53.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:53.575+0000 7fadb9d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:53.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:53.583+0000 7fadb9d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:53.992 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:53.992 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4898/44031 objects degraded (11.124%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":118,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":32,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":14677,"data_bytes":962134603,"bytes_used":341839872,"bytes_avail":386205216768,"bytes_total":386547056640,"degraded_objects":4898,"degraded_total":44031,"degraded_ratio":0.11123980831686767,"recovering_objects_per_sec":61,"recovering_bytes_per_sec":4037815,"recovering_keys_per_sec":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"read_bytes_sec":127,"write_bytes_sec":6470461,"read_op_per_sec":0,"write_op_per_sec":211},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:12:54.010 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:12:54.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:54.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:54.145+0000 7f102314e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:54.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:54.154+0000 7f102314e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:54.484 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:54.485 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:54.504 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-22T03:12:53.245177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":962134603,"num_objects":14677,"num_object_clones":0,"num_object_copies":44031,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":16618,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16566,"ondisk_log_size":16566,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":333828,"kb_used_data":160644,"kb_used_omap":0,"kb_used_meta":173120,"kb_avail":377153532,"statfs":{"total":386547056640,"available":386205216768,"internally_reserved":0,"allocated":164499456,"data_stored":2614269711,"data_compressed":124126818,"data_compressed_allocated":163381248,"data_compressed_original":2613854208,"omap_allocated":0,"internal_metadata":177274880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7405544,"num_objects":-112,"num_object_clones":0,"num_object_copies":-336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1694,"num_write_kb":50561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001665"},"pg_stats":[{"pgid":"3.f","version":"32'1121","reported_seq":1169,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186626+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:52.186626+0000","last_peered":"2024-09-22T03:12:52.186626+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:52.186626+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"31'1005","reported_seq":1053,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.725342+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:47.725342+0000","last_peered":"2024-09-22T03:12:47.725342+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:47.725342+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62783488,"num_objects":958,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1062","reported_seq":1108,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.164836+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:52.164836+0000","last_peered":"2024-09-22T03:12:52.164836+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:52.164836+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492056,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1063,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1103","reported_seq":1161,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.125973+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:52.125973+0000","last_peered":"2024-09-22T03:12:52.125973+0000","last_clean":"2024-09-22T03:12:52.125973+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:52.125973+0000","last_undegraded":"2024-09-22T03:12:52.125973+0000","last_fullsized":"2024-09-22T03:12:52.125973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1103,"log_dups_size":0,"ondisk_log_size":1103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58589184,"num_objects":894,"num_object_clones":0,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1104,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"31'991","reported_seq":1046,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.695962+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:47.695962+0000","last_peered":"2024-09-22T03:12:47.695962+0000","last_clean":"2024-09-22T03:12:47.695962+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:47.695962+0000","last_undegraded":"2024-09-22T03:12:47.695962+0000","last_fullsized":"2024-09-22T03:12:47.695962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":960,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1065","reported_seq":2115,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.098973+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:12:52.098973+0000","last_peered":"2024-09-22T03:12:52.098973+0000","last_clean":"2024-09-22T03:12:52.098973+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:12:52.098973+0000","last_undegraded":"2024-09-22T03:12:52.098973+0000","last_fullsized":"2024-09-22T03:12:52.098973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1065,"log_dups_size":0,"ondisk_log_size":1065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492032,"num_objects":862,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1066,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1080","reported_seq":1135,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.118209+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:52.118209+0000","last_peered":"2024-09-22T03:12:52.118209+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:52.118209+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55377920,"num_objects":845,"num_object_clones":0,"num_object_copies":2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1081,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'1031","reported_seq":1585,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.248276+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:48.248276+0000","last_peered":"2024-09-22T03:12:48.248276+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:48.248276+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64684032,"num_objects":987,"num_object_clones":0,"num_object_copies":2961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1031,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":738}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.363044+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:47.363044+0000","last_peered":"2024-09-22T03:12:47.363044+0000","last_clean":"2024-09-22T03:12:47.363044+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:47.363044+0000","last_undegraded":"2024-09-22T03:12:47.363044+0000","last_fullsized":"2024-09-22T03:12:47.363044+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1114","reported_seq":1157,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186620+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:52.186620+0000","last_peered":"2024-09-22T03:12:52.186620+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:52.186620+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57344000,"num_objects":875,"num_object_clones":0,"num_object_copies":2625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.830967+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:47.830967+0000","last_peered":"2024-09-22T03:12:47.830967+0000","last_clean":"2024-09-22T03:12:47.830967+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:47.830967+0000","last_undegraded":"2024-09-22T03:12:47.830967+0000","last_fullsized":"2024-09-22T03:12:47.830967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60686336,"num_objects":926,"num_object_clones":0,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":76,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.367365+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:47.367365+0000","last_peered":"2024-09-22T03:12:47.367365+0000","last_clean":"2024-09-22T03:12:47.367365+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:47.367365+0000","last_undegraded":"2024-09-22T03:12:47.367365+0000","last_fullsized":"2024-09-22T03:12:47.367365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"31'1005","reported_seq":1056,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.940948+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:47.940948+0000","last_peered":"2024-09-22T03:12:47.940948+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:47.940948+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.380545+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:47.380545+0000","last_peered":"2024-09-22T03:12:47.380545+0000","last_clean":"2024-09-22T03:12:47.380545+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:47.380545+0000","last_undegraded":"2024-09-22T03:12:47.380545+0000","last_fullsized":"2024-09-22T03:12:47.380545+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"31'989","reported_seq":1035,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.682510+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:47.682510+0000","last_peered":"2024-09-22T03:12:47.682510+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:47.682510+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60948480,"num_objects":930,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":990,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'977","reported_seq":1030,"reported_epoch":31,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:47.727320+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:47.727320+0000","last_peered":"2024-09-22T03:12:47.727320+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:47.727320+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61210624,"num_objects":934,"num_object_clones":0,"num_object_copies":2802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356684+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:48.356684+0000","last_peered":"2024-09-22T03:12:48.356684+0000","last_clean":"2024-09-22T03:12:48.356684+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:48.356684+0000","last_undegraded":"2024-09-22T03:12:48.356684+0000","last_fullsized":"2024-09-22T03:12:48.356684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'966","reported_seq":1085,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.251580+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:12:48.251580+0000","last_peered":"2024-09-22T03:12:48.251580+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:48.251580+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59244544,"num_objects":904,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":966,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":458}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.368991+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:47.368991+0000","last_peered":"2024-09-22T03:12:47.368991+0000","last_clean":"2024-09-22T03:12:47.368991+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:47.368991+0000","last_undegraded":"2024-09-22T03:12:47.368991+0000","last_fullsized":"2024-09-22T03:12:47.368991+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'975","reported_seq":1030,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:12:47.816973+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:47.816973+0000","last_peered":"2024-09-22T03:12:47.816973+0000","last_clean":"2024-09-22T03:12:47.816973+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:47.816973+0000","last_undegraded":"2024-09-22T03:12:47.816973+0000","last_fullsized":"2024-09-22T03:12:47.816973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356390+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:48.356390+0000","last_peered":"2024-09-22T03:12:48.356390+0000","last_clean":"2024-09-22T03:12:48.356390+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:48.356390+0000","last_undegraded":"2024-09-22T03:12:48.356390+0000","last_fullsized":"2024-09-22T03:12:48.356390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"31'1007","reported_seq":1609,"reported_epoch":31,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:48.246063+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:48.246063+0000","last_peered":"2024-09-22T03:12:48.246063+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:48.246063+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":18350080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":725}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356172+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:48.356172+0000","last_peered":"2024-09-22T03:12:48.356172+0000","last_clean":"2024-09-22T03:12:48.356172+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:48.356172+0000","last_undegraded":"2024-09-22T03:12:48.356172+0000","last_fullsized":"2024-09-22T03:12:48.356172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":961544216,"num_objects":14673,"num_object_clones":0,"num_object_copies":44019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16479,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1076,"num_bytes_recovered":70516736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141041664,"data_stored":2256535600,"data_compressed":107150624,"data_compressed_allocated":141033472,"data_compressed_original":2256535552,"omap_allocated":0,"internal_metadata":0},"log_size":16466,"ondisk_log_size":16466,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98772,"kb_used_data":53892,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94273068,"statfs":{"total":96636764160,"available":96535621632,"internally_reserved":0,"allocated":55185408,"data_stored":878925348,"data_compressed":41731290,"data_compressed_allocated":54927360,"data_compressed_original":878837760,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77584,"kb_used_data":42432,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94294256,"statfs":{"total":96636764160,"available":96557318144,"internally_reserved":0,"allocated":43450368,"data_stored":690837548,"data_compressed":32801943,"data_compressed_allocated":43175936,"data_compressed_original":690733056,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149716,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64964,"kb_used_data":25012,"kb_used_omap":0,"kb_used_meta":39936,"kb_avail":94306876,"statfs":{"total":96636764160,"available":96570241024,"internally_reserved":0,"allocated":25612288,"data_stored":405107817,"data_compressed":19234061,"data_compressed_allocated":25317376,"data_compressed_original":404996096,"omap_allocated":0,"internal_metadata":40894464},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92508,"kb_used_data":39308,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94279332,"statfs":{"total":96636764160,"available":96542035968,"internally_reserved":0,"allocated":40251392,"data_stored":639398998,"data_compressed":30359524,"data_compressed_allocated":39960576,"data_compressed_original":639287296,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39006208,"data_stored":624033816,"data_compressed":29632464,"data_compressed_allocated":39002112,"data_compressed_original":624033792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42733568,"data_stored":683737088,"data_compressed":32467093,"data_compressed_allocated":42733568,"data_compressed_original":683737088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":54423552,"data_stored":870711320,"data_compressed":41345526,"data_compressed_allocated":54419456,"data_compressed_original":870711296,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:54.605 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:54.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:54.751+0000 7f9902ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:54.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:54.761+0000 7f9902ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:56.939 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:12:57.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:57.087+0000 7f4050e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:57.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:57.093+0000 7f4050e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:57.507 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:57.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:57.656+0000 7ff980397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:57.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:57.666+0000 7ff980397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:58.017 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:58.019 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:58.040 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-22T03:12:57.246076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":898826827,"num_objects":13711,"num_object_clones":0,"num_object_copies":41133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":17584,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":82837504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17530,"ondisk_log_size":17530,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":334800,"kb_used_data":147720,"kb_used_omap":0,"kb_used_meta":187008,"kb_avail":377152560,"statfs":{"total":386547056640,"available":386204221440,"internally_reserved":0,"allocated":151265280,"data_stored":2402013017,"data_compressed":114048242,"data_compressed_allocated":150114304,"data_compressed_original":2401583104,"omap_allocated":0,"internal_metadata":191496192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-88342528,"num_objects":-1348,"num_object_clones":0,"num_object_copies":-4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2170,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.906454"},"pg_stats":[{"pgid":"3.f","version":"32'1121","reported_seq":1169,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186626+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:52.186626+0000","last_peered":"2024-09-22T03:12:52.186626+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:52.186626+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"32'1092","reported_seq":1142,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.113869+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:53.113869+0000","last_peered":"2024-09-22T03:12:53.113869+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:53.113869+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57081856,"num_objects":871,"num_object_clones":0,"num_object_copies":2613,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1093,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":336}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1062","reported_seq":1108,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.164836+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:52.164836+0000","last_peered":"2024-09-22T03:12:52.164836+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:52.164836+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492056,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1063,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1103","reported_seq":1161,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.125973+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:52.125973+0000","last_peered":"2024-09-22T03:12:52.125973+0000","last_clean":"2024-09-22T03:12:52.125973+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:52.125973+0000","last_undegraded":"2024-09-22T03:12:52.125973+0000","last_fullsized":"2024-09-22T03:12:52.125973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1103,"log_dups_size":0,"ondisk_log_size":1103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58589184,"num_objects":894,"num_object_clones":0,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1104,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"32'1094","reported_seq":1152,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:53.121135+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:53.121135+0000","last_peered":"2024-09-22T03:12:53.121135+0000","last_clean":"2024-09-22T03:12:53.121135+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:53.121135+0000","last_undegraded":"2024-09-22T03:12:53.121135+0000","last_fullsized":"2024-09-22T03:12:53.121135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1095,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1065","reported_seq":2115,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.098973+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:12:52.098973+0000","last_peered":"2024-09-22T03:12:52.098973+0000","last_clean":"2024-09-22T03:12:52.098973+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:12:52.098973+0000","last_undegraded":"2024-09-22T03:12:52.098973+0000","last_fullsized":"2024-09-22T03:12:52.098973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1065,"log_dups_size":0,"ondisk_log_size":1065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492032,"num_objects":862,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1066,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1080","reported_seq":1135,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.118209+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:52.118209+0000","last_peered":"2024-09-22T03:12:52.118209+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:52.118209+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55377920,"num_objects":845,"num_object_clones":0,"num_object_copies":2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1081,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"32'1118","reported_seq":1799,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.228512+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:53.228512+0000","last_peered":"2024-09-22T03:12:53.228512+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:53.228512+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58916864,"num_objects":899,"num_object_clones":0,"num_object_copies":2697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1119,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":20840448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":712}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357718+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:48.357718+0000","last_peered":"2024-09-22T03:12:48.357718+0000","last_clean":"2024-09-22T03:12:48.357718+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:48.357718+0000","last_undegraded":"2024-09-22T03:12:48.357718+0000","last_fullsized":"2024-09-22T03:12:48.357718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1114","reported_seq":1157,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186620+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:52.186620+0000","last_peered":"2024-09-22T03:12:52.186620+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:52.186620+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57344000,"num_objects":875,"num_object_clones":0,"num_object_copies":2625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"32'1077","reported_seq":1135,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:53.206428+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:53.206428+0000","last_peered":"2024-09-22T03:12:53.206428+0000","last_clean":"2024-09-22T03:12:53.206428+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:53.206428+0000","last_undegraded":"2024-09-22T03:12:53.206428+0000","last_fullsized":"2024-09-22T03:12:53.206428+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54001664,"num_objects":824,"num_object_clones":0,"num_object_copies":2472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1078,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.354822+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:48.354822+0000","last_peered":"2024-09-22T03:12:48.354822+0000","last_clean":"2024-09-22T03:12:48.354822+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:48.354822+0000","last_undegraded":"2024-09-22T03:12:48.354822+0000","last_fullsized":"2024-09-22T03:12:48.354822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"32'1108","reported_seq":1161,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.135805+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:53.135805+0000","last_peered":"2024-09-22T03:12:53.135805+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:53.135805+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55508992,"num_objects":847,"num_object_clones":0,"num_object_copies":2541,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1109,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":355}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.372759+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:48.372759+0000","last_peered":"2024-09-22T03:12:48.372759+0000","last_clean":"2024-09-22T03:12:48.372759+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:48.372759+0000","last_undegraded":"2024-09-22T03:12:48.372759+0000","last_fullsized":"2024-09-22T03:12:48.372759+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"32'1094","reported_seq":1142,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.140032+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:53.140032+0000","last_peered":"2024-09-22T03:12:53.140032+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:53.140032+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54067200,"num_objects":825,"num_object_clones":0,"num_object_copies":2475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1095,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":353}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'1074","reported_seq":1129,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.166233+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:53.166233+0000","last_peered":"2024-09-22T03:12:53.166233+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:53.166233+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1075,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":336}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356684+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:48.356684+0000","last_peered":"2024-09-22T03:12:48.356684+0000","last_clean":"2024-09-22T03:12:48.356684+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:48.356684+0000","last_undegraded":"2024-09-22T03:12:48.356684+0000","last_fullsized":"2024-09-22T03:12:48.356684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"32'1061","reported_seq":1304,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.237969+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:12:53.237969+0000","last_peered":"2024-09-22T03:12:53.237969+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:53.237969+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53018624,"num_objects":809,"num_object_clones":0,"num_object_copies":2427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":385,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1061,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":6356992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":424}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357252+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:48.357252+0000","last_peered":"2024-09-22T03:12:48.357252+0000","last_clean":"2024-09-22T03:12:48.357252+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:48.357252+0000","last_undegraded":"2024-09-22T03:12:48.357252+0000","last_fullsized":"2024-09-22T03:12:48.357252+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"32'1068","reported_seq":1126,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:53.223534+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:53.223534+0000","last_peered":"2024-09-22T03:12:53.223534+0000","last_clean":"2024-09-22T03:12:53.223534+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:53.223534+0000","last_undegraded":"2024-09-22T03:12:53.223534+0000","last_fullsized":"2024-09-22T03:12:53.223534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1068,"log_dups_size":0,"ondisk_log_size":1068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54329344,"num_objects":829,"num_object_clones":0,"num_object_copies":2487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1069,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356390+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:48.356390+0000","last_peered":"2024-09-22T03:12:48.356390+0000","last_clean":"2024-09-22T03:12:48.356390+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:48.356390+0000","last_undegraded":"2024-09-22T03:12:48.356390+0000","last_fullsized":"2024-09-22T03:12:48.356390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'1099","reported_seq":1834,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.235949+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:53.235949+0000","last_peered":"2024-09-22T03:12:53.235949+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:53.235949+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55574528,"num_objects":848,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1100,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":22609920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":697}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356172+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:48.356172+0000","last_peered":"2024-09-22T03:12:48.356172+0000","last_clean":"2024-09-22T03:12:48.356172+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:48.356172+0000","last_undegraded":"2024-09-22T03:12:48.356172+0000","last_fullsized":"2024-09-22T03:12:48.356172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":898236440,"num_objects":13707,"num_object_clones":0,"num_object_copies":41121,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17445,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":82837504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131047424,"data_stored":2096627760,"data_compressed":99558251,"data_compressed_allocated":131039232,"data_compressed_original":2096627712,"omap_allocated":0,"internal_metadata":0},"log_size":17430,"ondisk_log_size":17430,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98068,"kb_used_data":49024,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94273772,"statfs":{"total":96636764160,"available":96536342528,"internally_reserved":0,"allocated":50200576,"data_stored":798913097,"data_compressed":37932167,"data_compressed_allocated":49926144,"data_compressed_original":798818304,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95920,"kb_used_data":38560,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94275920,"statfs":{"total":96636764160,"available":96538542080,"internally_reserved":0,"allocated":39485440,"data_stored":627143761,"data_compressed":29777579,"data_compressed_allocated":39194624,"data_compressed_original":627032064,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149721,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48304,"kb_used_data":20828,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":94323536,"statfs":{"total":96636764160,"available":96587300864,"internally_reserved":0,"allocated":21327872,"data_stored":336557161,"data_compressed":15978972,"data_compressed_allocated":21032960,"data_compressed_original":336445440,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92508,"kb_used_data":39308,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94279332,"statfs":{"total":96636764160,"available":96542035968,"internally_reserved":0,"allocated":40251392,"data_stored":639398998,"data_compressed":30359524,"data_compressed_allocated":39960576,"data_compressed_original":639287296,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39006208,"data_stored":624033816,"data_compressed":29632464,"data_compressed_allocated":39002112,"data_compressed_original":624033792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38338560,"data_stored":613416960,"data_compressed":29128319,"data_compressed_allocated":38338560,"data_compressed_original":613416960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48824320,"data_stored":781123608,"data_compressed":37091927,"data_compressed_allocated":48820224,"data_compressed_original":781123584,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:58.042 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:12:58.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:58.191+0000 7fda4a24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:58.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:58.198+0000 7fda4a24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:58.612 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:58.612 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4710/41133 objects degraded (11.451%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":123,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":32,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":13711,"data_bytes":898826827,"bytes_used":342835200,"bytes_avail":386204221440,"bytes_total":386547056640,"degraded_objects":4710,"degraded_total":41133,"degraded_ratio":0.11450660053971264,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":3287895,"recovering_keys_per_sec":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"write_bytes_sec":2718964,"read_op_per_sec":0,"write_op_per_sec":219},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:12:58.630 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:12:58.631 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:12:58.794 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:58.785+0000 7fbf41757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:58.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:58.798+0000 7fbf41757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:59.093 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:12:59.094 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:12:59.114 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-22T03:12:57.246076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":898826827,"num_objects":13711,"num_object_clones":0,"num_object_copies":41133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":17584,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":82837504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17530,"ondisk_log_size":17530,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":334800,"kb_used_data":147720,"kb_used_omap":0,"kb_used_meta":187008,"kb_avail":377152560,"statfs":{"total":386547056640,"available":386204221440,"internally_reserved":0,"allocated":151265280,"data_stored":2402013017,"data_compressed":114048242,"data_compressed_allocated":150114304,"data_compressed_original":2401583104,"omap_allocated":0,"internal_metadata":191496192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-88342528,"num_objects":-1348,"num_object_clones":0,"num_object_copies":-4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2170,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.906454"},"pg_stats":[{"pgid":"3.f","version":"32'1121","reported_seq":1169,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186626+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:52.186626+0000","last_peered":"2024-09-22T03:12:52.186626+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:52.186626+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60424192,"num_objects":922,"num_object_clones":0,"num_object_copies":2766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"32'1092","reported_seq":1142,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.113869+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:53.113869+0000","last_peered":"2024-09-22T03:12:53.113869+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:53.113869+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57081856,"num_objects":871,"num_object_clones":0,"num_object_copies":2613,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1093,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":336}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1062","reported_seq":1108,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.164836+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:52.164836+0000","last_peered":"2024-09-22T03:12:52.164836+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:52.164836+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492056,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1063,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1103","reported_seq":1161,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.125973+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:52.125973+0000","last_peered":"2024-09-22T03:12:52.125973+0000","last_clean":"2024-09-22T03:12:52.125973+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:52.125973+0000","last_undegraded":"2024-09-22T03:12:52.125973+0000","last_fullsized":"2024-09-22T03:12:52.125973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1103,"log_dups_size":0,"ondisk_log_size":1103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58589184,"num_objects":894,"num_object_clones":0,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1104,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"32'1094","reported_seq":1152,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:53.121135+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:53.121135+0000","last_peered":"2024-09-22T03:12:53.121135+0000","last_clean":"2024-09-22T03:12:53.121135+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:53.121135+0000","last_undegraded":"2024-09-22T03:12:53.121135+0000","last_fullsized":"2024-09-22T03:12:53.121135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1095,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1065","reported_seq":2115,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:52.098973+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:12:52.098973+0000","last_peered":"2024-09-22T03:12:52.098973+0000","last_clean":"2024-09-22T03:12:52.098973+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:12:52.098973+0000","last_undegraded":"2024-09-22T03:12:52.098973+0000","last_fullsized":"2024-09-22T03:12:52.098973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1065,"log_dups_size":0,"ondisk_log_size":1065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492032,"num_objects":862,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1066,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1080","reported_seq":1135,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.118209+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:52.118209+0000","last_peered":"2024-09-22T03:12:52.118209+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:52.118209+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55377920,"num_objects":845,"num_object_clones":0,"num_object_copies":2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1081,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"32'1118","reported_seq":1799,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.228512+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:53.228512+0000","last_peered":"2024-09-22T03:12:53.228512+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:53.228512+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58916864,"num_objects":899,"num_object_clones":0,"num_object_copies":2697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1119,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":20840448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":712}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357718+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:48.357718+0000","last_peered":"2024-09-22T03:12:48.357718+0000","last_clean":"2024-09-22T03:12:48.357718+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:48.357718+0000","last_undegraded":"2024-09-22T03:12:48.357718+0000","last_fullsized":"2024-09-22T03:12:48.357718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1114","reported_seq":1157,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:52.186620+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:52.186620+0000","last_peered":"2024-09-22T03:12:52.186620+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:52.186620+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57344000,"num_objects":875,"num_object_clones":0,"num_object_copies":2625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"32'1077","reported_seq":1135,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:53.206428+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:53.206428+0000","last_peered":"2024-09-22T03:12:53.206428+0000","last_clean":"2024-09-22T03:12:53.206428+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:53.206428+0000","last_undegraded":"2024-09-22T03:12:53.206428+0000","last_fullsized":"2024-09-22T03:12:53.206428+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54001664,"num_objects":824,"num_object_clones":0,"num_object_copies":2472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1078,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.354822+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:48.354822+0000","last_peered":"2024-09-22T03:12:48.354822+0000","last_clean":"2024-09-22T03:12:48.354822+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:48.354822+0000","last_undegraded":"2024-09-22T03:12:48.354822+0000","last_fullsized":"2024-09-22T03:12:48.354822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"32'1108","reported_seq":1161,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.135805+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:53.135805+0000","last_peered":"2024-09-22T03:12:53.135805+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:53.135805+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55508992,"num_objects":847,"num_object_clones":0,"num_object_copies":2541,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1109,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":355}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.372759+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:48.372759+0000","last_peered":"2024-09-22T03:12:48.372759+0000","last_clean":"2024-09-22T03:12:48.372759+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:48.372759+0000","last_undegraded":"2024-09-22T03:12:48.372759+0000","last_fullsized":"2024-09-22T03:12:48.372759+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"32'1094","reported_seq":1142,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.140032+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:53.140032+0000","last_peered":"2024-09-22T03:12:53.140032+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:53.140032+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54067200,"num_objects":825,"num_object_clones":0,"num_object_copies":2475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1095,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":353}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'1074","reported_seq":1129,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.166233+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:53.166233+0000","last_peered":"2024-09-22T03:12:53.166233+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:53.166233+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1075,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":336}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356684+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:48.356684+0000","last_peered":"2024-09-22T03:12:48.356684+0000","last_clean":"2024-09-22T03:12:48.356684+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:48.356684+0000","last_undegraded":"2024-09-22T03:12:48.356684+0000","last_fullsized":"2024-09-22T03:12:48.356684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"32'1061","reported_seq":1304,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.237969+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:12:53.237969+0000","last_peered":"2024-09-22T03:12:53.237969+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:53.237969+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53018624,"num_objects":809,"num_object_clones":0,"num_object_copies":2427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":385,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1061,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":6356992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":424}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357252+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:48.357252+0000","last_peered":"2024-09-22T03:12:48.357252+0000","last_clean":"2024-09-22T03:12:48.357252+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:48.357252+0000","last_undegraded":"2024-09-22T03:12:48.357252+0000","last_fullsized":"2024-09-22T03:12:48.357252+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"32'1068","reported_seq":1126,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:53.223534+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:53.223534+0000","last_peered":"2024-09-22T03:12:53.223534+0000","last_clean":"2024-09-22T03:12:53.223534+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:53.223534+0000","last_undegraded":"2024-09-22T03:12:53.223534+0000","last_fullsized":"2024-09-22T03:12:53.223534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1068,"log_dups_size":0,"ondisk_log_size":1068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54329344,"num_objects":829,"num_object_clones":0,"num_object_copies":2487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1069,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356390+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:48.356390+0000","last_peered":"2024-09-22T03:12:48.356390+0000","last_clean":"2024-09-22T03:12:48.356390+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:48.356390+0000","last_undegraded":"2024-09-22T03:12:48.356390+0000","last_fullsized":"2024-09-22T03:12:48.356390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'1099","reported_seq":1834,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:53.235949+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:53.235949+0000","last_peered":"2024-09-22T03:12:53.235949+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:53.235949+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55574528,"num_objects":848,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1100,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":22609920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":697}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356172+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:48.356172+0000","last_peered":"2024-09-22T03:12:48.356172+0000","last_clean":"2024-09-22T03:12:48.356172+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:48.356172+0000","last_undegraded":"2024-09-22T03:12:48.356172+0000","last_fullsized":"2024-09-22T03:12:48.356172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":898236440,"num_objects":13707,"num_object_clones":0,"num_object_copies":41121,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17445,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":82837504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131047424,"data_stored":2096627760,"data_compressed":99558251,"data_compressed_allocated":131039232,"data_compressed_original":2096627712,"omap_allocated":0,"internal_metadata":0},"log_size":17430,"ondisk_log_size":17430,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98068,"kb_used_data":49024,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94273772,"statfs":{"total":96636764160,"available":96536342528,"internally_reserved":0,"allocated":50200576,"data_stored":798913097,"data_compressed":37932167,"data_compressed_allocated":49926144,"data_compressed_original":798818304,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95920,"kb_used_data":38560,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94275920,"statfs":{"total":96636764160,"available":96538542080,"internally_reserved":0,"allocated":39485440,"data_stored":627143761,"data_compressed":29777579,"data_compressed_allocated":39194624,"data_compressed_original":627032064,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149721,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48304,"kb_used_data":20828,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":94323536,"statfs":{"total":96636764160,"available":96587300864,"internally_reserved":0,"allocated":21327872,"data_stored":336557161,"data_compressed":15978972,"data_compressed_allocated":21032960,"data_compressed_original":336445440,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92508,"kb_used_data":39308,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94279332,"statfs":{"total":96636764160,"available":96542035968,"internally_reserved":0,"allocated":40251392,"data_stored":639398998,"data_compressed":30359524,"data_compressed_allocated":39960576,"data_compressed_original":639287296,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39006208,"data_stored":624033816,"data_compressed":29632464,"data_compressed_allocated":39002112,"data_compressed_original":624033792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38338560,"data_stored":613416960,"data_compressed":29128319,"data_compressed_allocated":38338560,"data_compressed_original":613416960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48824320,"data_stored":781123608,"data_compressed":37091927,"data_compressed_allocated":48820224,"data_compressed_original":781123584,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:12:59.279 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:12:59.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:59.431+0000 7f995d197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:12:59.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:12:59.438+0000 7f995d197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:01.591 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:01.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:01.720+0000 7f5cfc06c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:01.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:01.726+0000 7f5cfc06c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:02.116 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:02.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:02.256+0000 7f16dbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:02.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:02.267+0000 7f16dbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:02.519 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:02.519 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:02.537 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-22T03:13:01.246909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":794034763,"num_objects":12112,"num_object_clones":0,"num_object_copies":36336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":19183,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1430,"num_bytes_recovered":93716480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19129,"ondisk_log_size":19129,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":321552,"kb_used_data":130320,"kb_used_omap":0,"kb_used_meta":191168,"kb_avail":377165808,"statfs":{"total":386547056640,"available":386217787392,"internally_reserved":0,"allocated":133447680,"data_stored":2116931417,"data_compressed":100511042,"data_compressed_allocated":132296704,"data_compressed_original":2116501504,"omap_allocated":0,"internal_metadata":195756032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-203161600,"num_objects":-3100,"num_object_clones":0,"num_object_copies":-9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":23199744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"3.f","version":"32'1228","reported_seq":1276,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:57.001878+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:12:57.001878+0000","last_peered":"2024-09-22T03:12:57.001878+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:12:57.001878+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1228,"log_dups_size":0,"ondisk_log_size":1228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53411840,"num_objects":815,"num_object_clones":0,"num_object_copies":2445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1229,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":280}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"32'1195","reported_seq":1245,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:58.187692+0000","last_change":"2024-09-22T03:12:32.543831+0000","last_active":"2024-09-22T03:12:58.187692+0000","last_peered":"2024-09-22T03:12:58.187692+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:12:58.187692+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":768,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1196,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1178","reported_seq":1224,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:57.221437+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:12:57.221437+0000","last_peered":"2024-09-22T03:12:57.221437+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:12:57.221437+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1178,"log_dups_size":0,"ondisk_log_size":1178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889880,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1179,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1191","reported_seq":1249,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:57.183216+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:12:57.183216+0000","last_peered":"2024-09-22T03:12:57.183216+0000","last_clean":"2024-09-22T03:12:57.183216+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:12:57.183216+0000","last_undegraded":"2024-09-22T03:12:57.183216+0000","last_fullsized":"2024-09-22T03:12:57.183216+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1191,"log_dups_size":0,"ondisk_log_size":1191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52822016,"num_objects":806,"num_object_clones":0,"num_object_copies":2418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1192,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"32'1211","reported_seq":1269,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:58.218464+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:12:58.218464+0000","last_peered":"2024-09-22T03:12:58.218464+0000","last_clean":"2024-09-22T03:12:58.218464+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:12:58.218464+0000","last_undegraded":"2024-09-22T03:12:58.218464+0000","last_fullsized":"2024-09-22T03:12:58.218464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1211,"log_dups_size":0,"ondisk_log_size":1211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48496640,"num_objects":740,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1212,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1156","reported_seq":2206,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:57.175931+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:12:57.175931+0000","last_peered":"2024-09-22T03:12:57.175931+0000","last_clean":"2024-09-22T03:12:57.175931+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:12:57.175931+0000","last_undegraded":"2024-09-22T03:12:57.175931+0000","last_fullsized":"2024-09-22T03:12:57.175931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50528256,"num_objects":771,"num_object_clones":0,"num_object_copies":2313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1157,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1181","reported_seq":1236,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:57.159870+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:12:57.159870+0000","last_peered":"2024-09-22T03:12:57.159870+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:12:57.159870+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48758784,"num_objects":744,"num_object_clones":0,"num_object_copies":2232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1182,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"32'1230","reported_seq":2018,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:58.184332+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:12:58.184332+0000","last_peered":"2024-09-22T03:12:58.184332+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:12:58.184332+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1230,"log_dups_size":0,"ondisk_log_size":1230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51642368,"num_objects":788,"num_object_clones":0,"num_object_copies":2364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1230,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":24379392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":655}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357718+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:48.357718+0000","last_peered":"2024-09-22T03:12:48.357718+0000","last_clean":"2024-09-22T03:12:48.357718+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:48.357718+0000","last_undegraded":"2024-09-22T03:12:48.357718+0000","last_fullsized":"2024-09-22T03:12:48.357718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1211","reported_seq":1254,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:57.169120+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:12:57.169120+0000","last_peered":"2024-09-22T03:12:57.169120+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:12:57.169120+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1211,"log_dups_size":0,"ondisk_log_size":1211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50987008,"num_objects":778,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1212,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"32'1165","reported_seq":1223,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:58.186672+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:12:58.186672+0000","last_peered":"2024-09-22T03:12:58.186672+0000","last_clean":"2024-09-22T03:12:58.186672+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:12:58.186672+0000","last_undegraded":"2024-09-22T03:12:58.186672+0000","last_fullsized":"2024-09-22T03:12:58.186672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1165,"log_dups_size":0,"ondisk_log_size":1165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1166,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.354822+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:48.354822+0000","last_peered":"2024-09-22T03:12:48.354822+0000","last_clean":"2024-09-22T03:12:48.354822+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:48.354822+0000","last_undegraded":"2024-09-22T03:12:48.354822+0000","last_fullsized":"2024-09-22T03:12:48.354822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"32'1204","reported_seq":1257,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:58.229100+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:12:58.229100+0000","last_peered":"2024-09-22T03:12:58.229100+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:12:58.229100+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49217536,"num_objects":751,"num_object_clones":0,"num_object_copies":2253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1205,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":259}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.372759+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:48.372759+0000","last_peered":"2024-09-22T03:12:48.372759+0000","last_clean":"2024-09-22T03:12:48.372759+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:48.372759+0000","last_undegraded":"2024-09-22T03:12:48.372759+0000","last_fullsized":"2024-09-22T03:12:48.372759+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"32'1179","reported_seq":1227,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:58.178280+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:12:58.178280+0000","last_peered":"2024-09-22T03:12:58.178280+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:12:58.178280+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1179,"log_dups_size":0,"ondisk_log_size":1179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48496640,"num_objects":740,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1180,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":268}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'1171","reported_seq":1226,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:58.218886+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:12:58.218886+0000","last_peered":"2024-09-22T03:12:58.218886+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:12:58.218886+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48496640,"num_objects":740,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1172,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356684+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:48.356684+0000","last_peered":"2024-09-22T03:12:48.356684+0000","last_clean":"2024-09-22T03:12:48.356684+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:48.356684+0000","last_undegraded":"2024-09-22T03:12:48.356684+0000","last_fullsized":"2024-09-22T03:12:48.356684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"32'1156","reported_seq":1508,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:58.205946+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:12:58.205946+0000","last_peered":"2024-09-22T03:12:58.205946+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:12:58.205946+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1157,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":9895936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":382}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357252+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:48.357252+0000","last_peered":"2024-09-22T03:12:48.357252+0000","last_clean":"2024-09-22T03:12:48.357252+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:48.357252+0000","last_undegraded":"2024-09-22T03:12:48.357252+0000","last_fullsized":"2024-09-22T03:12:48.357252+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"32'1168","reported_seq":1226,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:58.200748+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:12:58.200748+0000","last_peered":"2024-09-22T03:12:58.200748+0000","last_clean":"2024-09-22T03:12:58.200748+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:12:58.200748+0000","last_undegraded":"2024-09-22T03:12:58.200748+0000","last_fullsized":"2024-09-22T03:12:58.200748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47775744,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1169,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356390+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:48.356390+0000","last_peered":"2024-09-22T03:12:48.356390+0000","last_clean":"2024-09-22T03:12:48.356390+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:48.356390+0000","last_undegraded":"2024-09-22T03:12:48.356390+0000","last_fullsized":"2024-09-22T03:12:48.356390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'1205","reported_seq":2056,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:12:58.188197+0000","last_change":"2024-09-22T03:12:32.623420+0000","last_active":"2024-09-22T03:12:58.188197+0000","last_peered":"2024-09-22T03:12:58.188197+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:12:58.188197+0000","last_undegraded":"2024-09-22T03:12:32.143373+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48627712,"num_objects":742,"num_object_clones":0,"num_object_copies":2226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":93,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1206,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":26411008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":649}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356172+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:48.356172+0000","last_peered":"2024-09-22T03:12:48.356172+0000","last_clean":"2024-09-22T03:12:48.356172+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:48.356172+0000","last_undegraded":"2024-09-22T03:12:48.356172+0000","last_fullsized":"2024-09-22T03:12:48.356172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":793444376,"num_objects":12108,"num_object_clones":0,"num_object_copies":36324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19044,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1430,"num_bytes_recovered":93716480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":116219904,"data_stored":1859387440,"data_compressed":88292811,"data_compressed_allocated":116211712,"data_compressed_original":1859387392,"omap_allocated":0,"internal_metadata":0},"log_size":19029,"ondisk_log_size":19029,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100728,"kb_used_data":43368,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94271112,"statfs":{"total":96636764160,"available":96533618688,"internally_reserved":0,"allocated":44408832,"data_stored":706245193,"data_compressed":33531799,"data_compressed_allocated":44134400,"data_compressed_original":706150400,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79080,"kb_used_data":34200,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94292760,"statfs":{"total":96636764160,"available":96555786240,"internally_reserved":0,"allocated":35020800,"data_stored":555709521,"data_compressed":26385499,"data_compressed_allocated":34729984,"data_compressed_original":555597824,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149726,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49572,"kb_used_data":17940,"kb_used_omap":0,"kb_used_meta":31616,"kb_avail":94322268,"statfs":{"total":96636764160,"available":96586002432,"internally_reserved":0,"allocated":18370560,"data_stored":289240169,"data_compressed":13732108,"data_compressed_allocated":18075648,"data_compressed_original":289128448,"omap_allocated":0,"internal_metadata":32374784},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92172,"kb_used_data":34812,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94279668,"statfs":{"total":96636764160,"available":96542380032,"internally_reserved":0,"allocated":35647488,"data_stored":565736534,"data_compressed":26861636,"data_compressed_allocated":35356672,"data_compressed_original":565624832,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34492416,"data_stored":551813144,"data_compressed":26203040,"data_compressed_allocated":34488320,"data_compressed_original":551813120,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33816576,"data_stored":541065216,"data_compressed":25692671,"data_compressed_allocated":33816576,"data_compressed_original":541065216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":43032576,"data_stored":688455704,"data_compressed":32691559,"data_compressed_allocated":43028480,"data_compressed_original":688455680,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:02.539 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:02.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:02.679+0000 7fc867557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:02.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:02.687+0000 7fc867557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:03.154 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:03.154 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4544/36336 objects degraded (12.506%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":128,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":32,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":12112,"data_bytes":794034763,"bytes_used":329269248,"bytes_avail":386217787392,"bytes_total":386547056640,"degraded_objects":4544,"degraded_total":36336,"degraded_ratio":0.12505504183179217,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":1932898,"recovering_keys_per_sec":0,"num_objects_recovered":354,"num_bytes_recovered":23199744,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":258},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:03.175 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:03.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:03.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:03.316+0000 7faaa422c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:03.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:03.328+0000 7faaa422c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:03.646 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:03.646 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:03.662 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-09-22T03:13:03.379119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":724238923,"num_objects":11047,"num_object_clones":0,"num_object_copies":33141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":20248,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2154,"num_bytes_recovered":141164544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20195,"ondisk_log_size":20195,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":306552,"kb_used_data":119480,"kb_used_omap":0,"kb_used_meta":187008,"kb_avail":377180808,"statfs":{"total":386547056640,"available":386233147392,"internally_reserved":0,"allocated":122347520,"data_stored":1939328857,"data_compressed":92077522,"data_compressed_allocated":121196544,"data_compressed_original":1938898944,"omap_allocated":0,"internal_metadata":191496192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-237895680,"num_objects":-3630,"num_object_clones":0,"num_object_copies":-10890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3630,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1078,"num_bytes_recovered":70647808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.133942"},"pg_stats":[{"pgid":"3.f","version":"32'1297","reported_seq":1441,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.099487+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:02.099487+0000","last_peered":"2024-09-22T03:13:02.099487+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:02.099487+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1297,"log_dups_size":0,"ondisk_log_size":1297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":3145728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"32'1261","reported_seq":1467,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.247981+0000","last_change":"2024-09-22T03:13:02.541383+0000","last_active":"2024-09-22T03:13:03.247981+0000","last_peered":"2024-09-22T03:13:03.247981+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:13:03.247981+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46071808,"num_objects":703,"num_object_clones":0,"num_object_copies":2109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":5177344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":246}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1240","reported_seq":1350,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.124304+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:02.124304+0000","last_peered":"2024-09-22T03:13:02.124304+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:02.124304+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1240,"log_dups_size":0,"ondisk_log_size":1240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44826648,"num_objects":685,"num_object_clones":0,"num_object_copies":2055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1241,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2162688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1267","reported_seq":1325,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:02.169444+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:02.169444+0000","last_peered":"2024-09-22T03:13:02.169444+0000","last_clean":"2024-09-22T03:13:02.169444+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:02.169444+0000","last_undegraded":"2024-09-22T03:13:02.169444+0000","last_fullsized":"2024-09-22T03:13:02.169444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1267,"log_dups_size":0,"ondisk_log_size":1267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1268,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"32'1278","reported_seq":1336,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:03.149173+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:03.149173+0000","last_peered":"2024-09-22T03:13:03.149173+0000","last_clean":"2024-09-22T03:13:03.149173+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:03.149173+0000","last_undegraded":"2024-09-22T03:13:03.149173+0000","last_fullsized":"2024-09-22T03:13:03.149173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44105728,"num_objects":673,"num_object_clones":0,"num_object_copies":2019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1279,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1222","reported_seq":2272,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:01.916224+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:01.916224+0000","last_peered":"2024-09-22T03:13:01.916224+0000","last_clean":"2024-09-22T03:13:01.916224+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:01.916224+0000","last_undegraded":"2024-09-22T03:13:01.916224+0000","last_fullsized":"2024-09-22T03:13:01.916224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1222,"log_dups_size":0,"ondisk_log_size":1222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46202880,"num_objects":705,"num_object_clones":0,"num_object_copies":2115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1223,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1274","reported_seq":1443,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.144480+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:02.144480+0000","last_peered":"2024-09-22T03:13:02.144480+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:02.144480+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42663936,"num_objects":651,"num_object_clones":0,"num_object_copies":1953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1275,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"32'1289","reported_seq":2264,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.250199+0000","last_change":"2024-09-22T03:12:32.683727+0000","last_active":"2024-09-22T03:13:03.250199+0000","last_peered":"2024-09-22T03:13:03.250199+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:13:03.250199+0000","last_undegraded":"2024-09-22T03:12:32.346652+0000","last_fullsized":"2024-09-22T03:12:32.222953+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1289,"log_dups_size":0,"ondisk_log_size":1289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47710208,"num_objects":728,"num_object_clones":0,"num_object_copies":2184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1290,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":465,"num_bytes_recovered":30474240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":688}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357718+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:48.357718+0000","last_peered":"2024-09-22T03:12:48.357718+0000","last_clean":"2024-09-22T03:12:48.357718+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:48.357718+0000","last_undegraded":"2024-09-22T03:12:48.357718+0000","last_fullsized":"2024-09-22T03:12:48.357718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1296","reported_seq":1437,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.229816+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:13:02.229816+0000","last_peered":"2024-09-22T03:13:02.229816+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:02.229816+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45416448,"num_objects":693,"num_object_clones":0,"num_object_copies":2079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1297,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"32'1229","reported_seq":1287,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:02.941232+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:02.941232+0000","last_peered":"2024-09-22T03:13:02.941232+0000","last_clean":"2024-09-22T03:13:02.941232+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:02.941232+0000","last_undegraded":"2024-09-22T03:13:02.941232+0000","last_fullsized":"2024-09-22T03:13:02.941232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44040192,"num_objects":672,"num_object_clones":0,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1230,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.354822+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:48.354822+0000","last_peered":"2024-09-22T03:12:48.354822+0000","last_clean":"2024-09-22T03:12:48.354822+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:48.354822+0000","last_undegraded":"2024-09-22T03:12:48.354822+0000","last_fullsized":"2024-09-22T03:12:48.354822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"32'1272","reported_seq":1445,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.163159+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:13:03.163159+0000","last_peered":"2024-09-22T03:13:03.163159+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:03.163159+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1272,"log_dups_size":0,"ondisk_log_size":1272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44761088,"num_objects":683,"num_object_clones":0,"num_object_copies":2049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1273,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3997696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":251}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.372759+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:48.372759+0000","last_peered":"2024-09-22T03:12:48.372759+0000","last_clean":"2024-09-22T03:12:48.372759+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:48.372759+0000","last_undegraded":"2024-09-22T03:12:48.372759+0000","last_fullsized":"2024-09-22T03:12:48.372759+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"32'1250","reported_seq":1428,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.208681+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:03.208681+0000","last_peered":"2024-09-22T03:13:03.208681+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:03.208681+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1250,"log_dups_size":0,"ondisk_log_size":1250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43843584,"num_objects":669,"num_object_clones":0,"num_object_copies":2007,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1251,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'1232","reported_seq":1397,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.205158+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:13:03.205158+0000","last_peered":"2024-09-22T03:13:03.205158+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:03.205158+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44498944,"num_objects":679,"num_object_clones":0,"num_object_copies":2037,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1233,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":3735552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356684+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:12:48.356684+0000","last_peered":"2024-09-22T03:12:48.356684+0000","last_clean":"2024-09-22T03:12:48.356684+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:12:48.356684+0000","last_undegraded":"2024-09-22T03:12:48.356684+0000","last_fullsized":"2024-09-22T03:12:48.356684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"32'1208","reported_seq":1747,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.203059+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:13:03.203059+0000","last_peered":"2024-09-22T03:13:03.203059+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:13:03.203059+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43384832,"num_objects":662,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":16056320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":425}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357252+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:48.357252+0000","last_peered":"2024-09-22T03:12:48.357252+0000","last_clean":"2024-09-22T03:12:48.357252+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:48.357252+0000","last_undegraded":"2024-09-22T03:12:48.357252+0000","last_fullsized":"2024-09-22T03:12:48.357252+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"32'1232","reported_seq":1290,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:03.110984+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:03.110984+0000","last_peered":"2024-09-22T03:13:03.110984+0000","last_clean":"2024-09-22T03:13:03.110984+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:03.110984+0000","last_undegraded":"2024-09-22T03:13:03.110984+0000","last_fullsized":"2024-09-22T03:13:03.110984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43581440,"num_objects":665,"num_object_clones":0,"num_object_copies":1995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1233,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356390+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:12:48.356390+0000","last_peered":"2024-09-22T03:12:48.356390+0000","last_clean":"2024-09-22T03:12:48.356390+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:12:48.356390+0000","last_undegraded":"2024-09-22T03:12:48.356390+0000","last_fullsized":"2024-09-22T03:12:48.356390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'1248","reported_seq":2286,"reported_epoch":32,"state":"active+undersized+remapped","last_fresh":"2024-09-22T03:13:02.696193+0000","last_change":"2024-09-22T03:13:02.509536+0000","last_active":"2024-09-22T03:13:02.696193+0000","last_peered":"2024-09-22T03:13:02.696193+0000","last_clean":"2024-09-22T03:11:38.278336+0000","last_became_active":"2024-09-22T03:12:32.545483+0000","last_became_peered":"2024-09-22T03:12:32.545483+0000","last_unstale":"2024-09-22T03:13:02.696193+0000","last_undegraded":"2024-09-22T03:13:02.696193+0000","last_fullsized":"2024-09-22T03:12:32.048718+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1248,"log_dups_size":0,"ondisk_log_size":1248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45809664,"num_objects":699,"num_object_clones":0,"num_object_copies":2097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1249,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[3,2],"avail_no_missing":["3","1","2"],"object_location_counts":[{"shards":"2,3","objects":699}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.356172+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:12:48.356172+0000","last_peered":"2024-09-22T03:12:48.356172+0000","last_clean":"2024-09-22T03:12:48.356172+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:12:48.356172+0000","last_undegraded":"2024-09-22T03:12:48.356172+0000","last_fullsized":"2024-09-22T03:12:48.356172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":723648536,"num_objects":11043,"num_object_clones":0,"num_object_copies":33129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20109,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2154,"num_bytes_recovered":141164544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106516480,"data_stored":1704132656,"data_compressed":80920483,"data_compressed_allocated":106508288,"data_compressed_original":1704132608,"omap_allocated":0,"internal_metadata":0},"log_size":20095,"ondisk_log_size":20095,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100884,"kb_used_data":39364,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94270956,"statfs":{"total":96636764160,"available":96533458944,"internally_reserved":0,"allocated":40308736,"data_stored":640643657,"data_compressed":30416687,"data_compressed_allocated":40034304,"data_compressed_original":640548864,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79956,"kb_used_data":30916,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94291884,"statfs":{"total":96636764160,"available":96554889216,"internally_reserved":0,"allocated":31657984,"data_stored":501904465,"data_compressed":23830547,"data_compressed_allocated":31367168,"data_compressed_original":501792768,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149726,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49572,"kb_used_data":17940,"kb_used_omap":0,"kb_used_meta":31616,"kb_avail":94322268,"statfs":{"total":96636764160,"available":96586002432,"internally_reserved":0,"allocated":18370560,"data_stored":289240169,"data_compressed":13732108,"data_compressed_allocated":18075648,"data_compressed_original":289128448,"omap_allocated":0,"internal_metadata":32374784},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76140,"kb_used_data":31260,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94295700,"statfs":{"total":96636764160,"available":96558796800,"internally_reserved":0,"allocated":32010240,"data_stored":507540566,"data_compressed":24098180,"data_compressed_allocated":31719424,"data_compressed_original":507428864,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31199232,"data_stored":499122200,"data_compressed":23700992,"data_compressed_allocated":31195136,"data_compressed_original":499122176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":78053376,"data_compressed":3705541,"data_compressed_allocated":4878336,"data_compressed_original":78053376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30965760,"data_stored":495452160,"data_compressed":23526719,"data_compressed_allocated":30965760,"data_compressed_original":495452160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":39473152,"data_stored":631504920,"data_compressed":29987231,"data_compressed_allocated":39469056,"data_compressed_original":631504896,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:03.868 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:03.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:03.992+0000 7f2da584f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:04.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:04.001+0000 7f2da584f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:06.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:06.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:06.299+0000 7f1916c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:06.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:06.305+0000 7f1916c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:06.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:06.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:06.821+0000 7fcad3f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:06.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:06.835+0000 7fcad3f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:07.197 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:07.198 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:07.216 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-22T03:13:06.013341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":721355339,"num_objects":11003,"num_object_clones":0,"num_object_copies":33009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":20292,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2194,"num_bytes_recovered":143785984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20240,"ondisk_log_size":20240,"up":75,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":329300,"kb_used_data":120020,"kb_used_omap":0,"kb_used_meta":209216,"kb_avail":377158060,"statfs":{"total":386547056640,"available":386209853440,"internally_reserved":0,"allocated":122900480,"data_stored":1947793703,"data_compressed":92478969,"data_compressed_allocated":121724928,"data_compressed_original":1947353088,"omap_allocated":0,"internal_metadata":214237184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-72679424,"num_objects":-1109,"num_object_clones":0,"num_object_copies":-3327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1109,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":764,"num_bytes_recovered":50069504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.766431"},"pg_stats":[{"pgid":"3.f","version":"32'1297","reported_seq":1441,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.099487+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:02.099487+0000","last_peered":"2024-09-22T03:13:02.099487+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:02.099487+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1297,"log_dups_size":0,"ondisk_log_size":1297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":3145728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"32'1261","reported_seq":1467,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.247981+0000","last_change":"2024-09-22T03:13:02.541383+0000","last_active":"2024-09-22T03:13:03.247981+0000","last_peered":"2024-09-22T03:13:03.247981+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:13:03.247981+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46071808,"num_objects":703,"num_object_clones":0,"num_object_copies":2109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":5177344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":246}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1240","reported_seq":1350,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.124304+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:02.124304+0000","last_peered":"2024-09-22T03:13:02.124304+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:02.124304+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1240,"log_dups_size":0,"ondisk_log_size":1240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44826648,"num_objects":685,"num_object_clones":0,"num_object_copies":2055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1241,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2162688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1267","reported_seq":1325,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:02.169444+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:02.169444+0000","last_peered":"2024-09-22T03:13:02.169444+0000","last_clean":"2024-09-22T03:13:02.169444+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:02.169444+0000","last_undegraded":"2024-09-22T03:13:02.169444+0000","last_fullsized":"2024-09-22T03:13:02.169444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1267,"log_dups_size":0,"ondisk_log_size":1267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1268,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"32'1278","reported_seq":1336,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:03.149173+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:03.149173+0000","last_peered":"2024-09-22T03:13:03.149173+0000","last_clean":"2024-09-22T03:13:03.149173+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:03.149173+0000","last_undegraded":"2024-09-22T03:13:03.149173+0000","last_fullsized":"2024-09-22T03:13:03.149173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44105728,"num_objects":673,"num_object_clones":0,"num_object_copies":2019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1279,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1222","reported_seq":2272,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:01.916224+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:01.916224+0000","last_peered":"2024-09-22T03:13:01.916224+0000","last_clean":"2024-09-22T03:13:01.916224+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:01.916224+0000","last_undegraded":"2024-09-22T03:13:01.916224+0000","last_fullsized":"2024-09-22T03:13:01.916224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1222,"log_dups_size":0,"ondisk_log_size":1222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46202880,"num_objects":705,"num_object_clones":0,"num_object_copies":2115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1223,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1274","reported_seq":1443,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.144480+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:02.144480+0000","last_peered":"2024-09-22T03:13:02.144480+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:02.144480+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42663936,"num_objects":651,"num_object_clones":0,"num_object_copies":1953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1275,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"34'1311","reported_seq":2373,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-22T03:13:05.067049+0000","last_change":"2024-09-22T03:13:05.054013+0000","last_active":"2024-09-22T03:13:05.054013+0000","last_peered":"2024-09-22T03:13:04.991131+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:13:05.067049+0000","last_undegraded":"2024-09-22T03:13:05.067049+0000","last_fullsized":"2024-09-22T03:13:05.067049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1311,"log_dups_size":0,"ondisk_log_size":1311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46333952,"num_objects":707,"num_object_clones":0,"num_object_copies":2121,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1311,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357718+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:48.357718+0000","last_peered":"2024-09-22T03:12:48.357718+0000","last_clean":"2024-09-22T03:12:48.357718+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:48.357718+0000","last_undegraded":"2024-09-22T03:12:48.357718+0000","last_fullsized":"2024-09-22T03:12:48.357718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1296","reported_seq":1437,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.229816+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:13:02.229816+0000","last_peered":"2024-09-22T03:13:02.229816+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:02.229816+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45416448,"num_objects":693,"num_object_clones":0,"num_object_copies":2079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1297,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"32'1229","reported_seq":1287,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:02.941232+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:02.941232+0000","last_peered":"2024-09-22T03:13:02.941232+0000","last_clean":"2024-09-22T03:13:02.941232+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:02.941232+0000","last_undegraded":"2024-09-22T03:13:02.941232+0000","last_fullsized":"2024-09-22T03:13:02.941232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44040192,"num_objects":672,"num_object_clones":0,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1230,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.354822+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:48.354822+0000","last_peered":"2024-09-22T03:12:48.354822+0000","last_clean":"2024-09-22T03:12:48.354822+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:48.354822+0000","last_undegraded":"2024-09-22T03:12:48.354822+0000","last_fullsized":"2024-09-22T03:12:48.354822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"32'1272","reported_seq":1445,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.163159+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:13:03.163159+0000","last_peered":"2024-09-22T03:13:03.163159+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:03.163159+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1272,"log_dups_size":0,"ondisk_log_size":1272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44761088,"num_objects":683,"num_object_clones":0,"num_object_copies":2049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1273,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3997696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":251}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.372759+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:48.372759+0000","last_peered":"2024-09-22T03:12:48.372759+0000","last_clean":"2024-09-22T03:12:48.372759+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:48.372759+0000","last_undegraded":"2024-09-22T03:12:48.372759+0000","last_fullsized":"2024-09-22T03:12:48.372759+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"32'1250","reported_seq":1428,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.208681+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:03.208681+0000","last_peered":"2024-09-22T03:13:03.208681+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:03.208681+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1250,"log_dups_size":0,"ondisk_log_size":1250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43843584,"num_objects":669,"num_object_clones":0,"num_object_copies":2007,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1251,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'1232","reported_seq":1397,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.205158+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:13:03.205158+0000","last_peered":"2024-09-22T03:13:03.205158+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:03.205158+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44498944,"num_objects":679,"num_object_clones":0,"num_object_copies":2037,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1233,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":3735552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.051581+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:05.051581+0000","last_peered":"2024-09-22T03:13:05.051581+0000","last_clean":"2024-09-22T03:13:05.051581+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:05.051581+0000","last_undegraded":"2024-09-22T03:13:05.051581+0000","last_fullsized":"2024-09-22T03:13:05.051581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"32'1208","reported_seq":1747,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.203059+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:13:03.203059+0000","last_peered":"2024-09-22T03:13:03.203059+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:13:03.203059+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43384832,"num_objects":662,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":16056320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":425}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357252+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:48.357252+0000","last_peered":"2024-09-22T03:12:48.357252+0000","last_clean":"2024-09-22T03:12:48.357252+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:48.357252+0000","last_undegraded":"2024-09-22T03:12:48.357252+0000","last_fullsized":"2024-09-22T03:12:48.357252+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"32'1232","reported_seq":1290,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:03.110984+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:03.110984+0000","last_peered":"2024-09-22T03:13:03.110984+0000","last_clean":"2024-09-22T03:13:03.110984+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:03.110984+0000","last_undegraded":"2024-09-22T03:13:03.110984+0000","last_fullsized":"2024-09-22T03:13:03.110984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43581440,"num_objects":665,"num_object_clones":0,"num_object_copies":1995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1233,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.056234+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:05.056234+0000","last_peered":"2024-09-22T03:13:05.056234+0000","last_clean":"2024-09-22T03:13:05.056234+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:05.056234+0000","last_undegraded":"2024-09-22T03:13:05.056234+0000","last_fullsized":"2024-09-22T03:13:05.056234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"35'1271","reported_seq":2323,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.257056+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:05.257056+0000","last_peered":"2024-09-22T03:13:05.257056+0000","last_clean":"2024-09-22T03:13:05.257056+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:05.257056+0000","last_undegraded":"2024-09-22T03:13:05.257056+0000","last_fullsized":"2024-09-22T03:13:05.257056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44302336,"num_objects":676,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1272,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":54,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.055218+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:05.055218+0000","last_peered":"2024-09-22T03:13:05.055218+0000","last_clean":"2024-09-22T03:13:05.055218+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:05.055218+0000","last_undegraded":"2024-09-22T03:13:05.055218+0000","last_fullsized":"2024-09-22T03:13:05.055218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":720764952,"num_objects":10999,"num_object_clones":0,"num_object_copies":32997,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20153,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2194,"num_bytes_recovered":143785984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120209408,"data_stored":1923153992,"data_compressed":91321637,"data_compressed_allocated":120197120,"data_compressed_original":1923153920,"omap_allocated":0,"internal_metadata":0},"log_size":20140,"ondisk_log_size":20140,"up":48,"acting":39,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100884,"kb_used_data":39364,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94270956,"statfs":{"total":96636764160,"available":96533458944,"internally_reserved":0,"allocated":40308736,"data_stored":640643657,"data_compressed":30416687,"data_compressed_allocated":40034304,"data_compressed_original":640548864,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79956,"kb_used_data":30916,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94291884,"statfs":{"total":96636764160,"available":96554889216,"internally_reserved":0,"allocated":31657984,"data_stored":501904465,"data_compressed":23830547,"data_compressed_allocated":31367168,"data_compressed_original":501792768,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149731,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72320,"kb_used_data":18480,"kb_used_omap":0,"kb_used_meta":53824,"kb_avail":94299520,"statfs":{"total":96636764160,"available":96562708480,"internally_reserved":0,"allocated":18923520,"data_stored":297705015,"data_compressed":14133555,"data_compressed_allocated":18604032,"data_compressed_original":297582592,"omap_allocated":0,"internal_metadata":55115776},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76140,"kb_used_data":31260,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94295700,"statfs":{"total":96636764160,"available":96558796800,"internally_reserved":0,"allocated":32010240,"data_stored":507540566,"data_compressed":24098180,"data_compressed_allocated":31719424,"data_compressed_original":507428864,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31199232,"data_stored":499122200,"data_compressed":23700992,"data_compressed_allocated":31195136,"data_compressed_original":499122176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18571264,"data_stored":297074712,"data_compressed":14106695,"data_compressed_allocated":18567168,"data_compressed_original":297074688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30965760,"data_stored":495452160,"data_compressed":23526719,"data_compressed_allocated":30965760,"data_compressed_original":495452160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":39473152,"data_stored":631504920,"data_compressed":29987231,"data_compressed_allocated":39469056,"data_compressed_original":631504896,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:07.218 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:07.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:07.375+0000 7fcd56197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:07.385+0000 7fcd56197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:07.766 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3820/33141 objects degraded (11.527%), 10 pgs degraded","count":10},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":132,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":36,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":3},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":25,"num_pools":3,"num_objects":11047,"data_bytes":724238923,"bytes_used":313909248,"bytes_avail":386233147392,"bytes_total":386547056640,"degraded_objects":3820,"degraded_total":33141,"degraded_ratio":0.11526507950876558,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":7906259,"recovering_keys_per_sec":0,"num_objects_recovered":724,"num_bytes_recovered":47448064,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":177},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:07.784 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:07.784 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:07.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:07.913+0000 7f0a29d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:07.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:07.922+0000 7f0a29d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:08.241 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:08.241 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:08.261 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-09-22T03:13:07.248222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":721355339,"num_objects":11003,"num_object_clones":0,"num_object_copies":33009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":20292,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2194,"num_bytes_recovered":143785984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20240,"ondisk_log_size":20240,"up":75,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":329300,"kb_used_data":120020,"kb_used_omap":0,"kb_used_meta":209216,"kb_avail":377158060,"statfs":{"total":386547056640,"available":386209853440,"internally_reserved":0,"allocated":122900480,"data_stored":1947793703,"data_compressed":92478969,"data_compressed_allocated":121724928,"data_compressed_original":1947353088,"omap_allocated":0,"internal_metadata":214237184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-43122688,"num_objects":-658,"num_object_clones":0,"num_object_copies":-1974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":658,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":578,"num_bytes_recovered":37879808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000847"},"pg_stats":[{"pgid":"3.f","version":"32'1297","reported_seq":1441,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.099487+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:02.099487+0000","last_peered":"2024-09-22T03:13:02.099487+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:02.099487+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1297,"log_dups_size":0,"ondisk_log_size":1297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":3145728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"32'1261","reported_seq":1467,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.247981+0000","last_change":"2024-09-22T03:13:02.541383+0000","last_active":"2024-09-22T03:13:03.247981+0000","last_peered":"2024-09-22T03:13:03.247981+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:13:03.247981+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46071808,"num_objects":703,"num_object_clones":0,"num_object_copies":2109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":5177344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":246}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"32'1240","reported_seq":1350,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.124304+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:02.124304+0000","last_peered":"2024-09-22T03:13:02.124304+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:02.124304+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1240,"log_dups_size":0,"ondisk_log_size":1240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44826648,"num_objects":685,"num_object_clones":0,"num_object_copies":2055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1241,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2162688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"32'1267","reported_seq":1325,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:02.169444+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:02.169444+0000","last_peered":"2024-09-22T03:13:02.169444+0000","last_clean":"2024-09-22T03:13:02.169444+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:02.169444+0000","last_undegraded":"2024-09-22T03:13:02.169444+0000","last_fullsized":"2024-09-22T03:13:02.169444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1267,"log_dups_size":0,"ondisk_log_size":1267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1268,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"32'1278","reported_seq":1336,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:03.149173+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:03.149173+0000","last_peered":"2024-09-22T03:13:03.149173+0000","last_clean":"2024-09-22T03:13:03.149173+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:03.149173+0000","last_undegraded":"2024-09-22T03:13:03.149173+0000","last_fullsized":"2024-09-22T03:13:03.149173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44105728,"num_objects":673,"num_object_clones":0,"num_object_copies":2019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1279,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'1222","reported_seq":2272,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:01.916224+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:01.916224+0000","last_peered":"2024-09-22T03:13:01.916224+0000","last_clean":"2024-09-22T03:13:01.916224+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:01.916224+0000","last_undegraded":"2024-09-22T03:13:01.916224+0000","last_fullsized":"2024-09-22T03:13:01.916224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1222,"log_dups_size":0,"ondisk_log_size":1222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46202880,"num_objects":705,"num_object_clones":0,"num_object_copies":2115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1223,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"32'1274","reported_seq":1443,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.144480+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:02.144480+0000","last_peered":"2024-09-22T03:13:02.144480+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:02.144480+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42663936,"num_objects":651,"num_object_clones":0,"num_object_copies":1953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1275,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"34'1311","reported_seq":2373,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-22T03:13:05.067049+0000","last_change":"2024-09-22T03:13:05.054013+0000","last_active":"2024-09-22T03:13:05.054013+0000","last_peered":"2024-09-22T03:13:04.991131+0000","last_clean":"2024-09-22T03:11:38.299159+0000","last_became_active":"2024-09-22T03:12:32.618564+0000","last_became_peered":"2024-09-22T03:12:32.618564+0000","last_unstale":"2024-09-22T03:13:05.067049+0000","last_undegraded":"2024-09-22T03:13:05.067049+0000","last_fullsized":"2024-09-22T03:13:05.067049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1311,"log_dups_size":0,"ondisk_log_size":1311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46333952,"num_objects":707,"num_object_clones":0,"num_object_copies":2121,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1311,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357718+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:12:48.357718+0000","last_peered":"2024-09-22T03:12:48.357718+0000","last_clean":"2024-09-22T03:12:48.357718+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:12:48.357718+0000","last_undegraded":"2024-09-22T03:12:48.357718+0000","last_fullsized":"2024-09-22T03:12:48.357718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'1296","reported_seq":1437,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:02.229816+0000","last_change":"2024-09-22T03:12:32.552097+0000","last_active":"2024-09-22T03:13:02.229816+0000","last_peered":"2024-09-22T03:13:02.229816+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:02.229816+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45416448,"num_objects":693,"num_object_clones":0,"num_object_copies":2079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1297,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.353573+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:12:48.353573+0000","last_peered":"2024-09-22T03:12:48.353573+0000","last_clean":"2024-09-22T03:12:48.353573+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:12:48.353573+0000","last_undegraded":"2024-09-22T03:12:48.353573+0000","last_fullsized":"2024-09-22T03:12:48.353573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"32'1229","reported_seq":1287,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:02.941232+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:02.941232+0000","last_peered":"2024-09-22T03:13:02.941232+0000","last_clean":"2024-09-22T03:13:02.941232+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:02.941232+0000","last_undegraded":"2024-09-22T03:13:02.941232+0000","last_fullsized":"2024-09-22T03:13:02.941232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44040192,"num_objects":672,"num_object_clones":0,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1230,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.354822+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:12:48.354822+0000","last_peered":"2024-09-22T03:12:48.354822+0000","last_clean":"2024-09-22T03:12:48.354822+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:12:48.354822+0000","last_undegraded":"2024-09-22T03:12:48.354822+0000","last_fullsized":"2024-09-22T03:12:48.354822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"32'1272","reported_seq":1445,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.163159+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:13:03.163159+0000","last_peered":"2024-09-22T03:13:03.163159+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:03.163159+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1272,"log_dups_size":0,"ondisk_log_size":1272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44761088,"num_objects":683,"num_object_clones":0,"num_object_copies":2049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1273,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3997696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":251}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.372759+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:12:48.372759+0000","last_peered":"2024-09-22T03:12:48.372759+0000","last_clean":"2024-09-22T03:12:48.372759+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:12:48.372759+0000","last_undegraded":"2024-09-22T03:12:48.372759+0000","last_fullsized":"2024-09-22T03:12:48.372759+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.363918+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:12:48.363918+0000","last_peered":"2024-09-22T03:12:48.363918+0000","last_clean":"2024-09-22T03:12:48.363918+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:12:48.363918+0000","last_undegraded":"2024-09-22T03:12:48.363918+0000","last_fullsized":"2024-09-22T03:12:48.363918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"32'1250","reported_seq":1428,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.208681+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:03.208681+0000","last_peered":"2024-09-22T03:13:03.208681+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:03.208681+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1250,"log_dups_size":0,"ondisk_log_size":1250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43843584,"num_objects":669,"num_object_clones":0,"num_object_copies":2007,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1251,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'1232","reported_seq":1397,"reported_epoch":32,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.205158+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:13:03.205158+0000","last_peered":"2024-09-22T03:13:03.205158+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:03.205158+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44498944,"num_objects":679,"num_object_clones":0,"num_object_copies":2037,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1233,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":3735552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.051581+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:05.051581+0000","last_peered":"2024-09-22T03:13:05.051581+0000","last_clean":"2024-09-22T03:13:05.051581+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:05.051581+0000","last_undegraded":"2024-09-22T03:13:05.051581+0000","last_fullsized":"2024-09-22T03:13:05.051581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"32'1208","reported_seq":1747,"reported_epoch":32,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:03.203059+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:13:03.203059+0000","last_peered":"2024-09-22T03:13:03.203059+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:13:03.203059+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43384832,"num_objects":662,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":16056320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":425}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:12:48.357252+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:12:48.357252+0000","last_peered":"2024-09-22T03:12:48.357252+0000","last_clean":"2024-09-22T03:12:48.357252+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:12:48.357252+0000","last_undegraded":"2024-09-22T03:12:48.357252+0000","last_fullsized":"2024-09-22T03:12:48.357252+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"32'1232","reported_seq":1290,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-22T03:13:03.110984+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:03.110984+0000","last_peered":"2024-09-22T03:13:03.110984+0000","last_clean":"2024-09-22T03:13:03.110984+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:03.110984+0000","last_undegraded":"2024-09-22T03:13:03.110984+0000","last_fullsized":"2024-09-22T03:13:03.110984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43581440,"num_objects":665,"num_object_clones":0,"num_object_copies":1995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1233,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.056234+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:05.056234+0000","last_peered":"2024-09-22T03:13:05.056234+0000","last_clean":"2024-09-22T03:13:05.056234+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:05.056234+0000","last_undegraded":"2024-09-22T03:13:05.056234+0000","last_fullsized":"2024-09-22T03:13:05.056234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"35'1271","reported_seq":2323,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.257056+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:05.257056+0000","last_peered":"2024-09-22T03:13:05.257056+0000","last_clean":"2024-09-22T03:13:05.257056+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:05.257056+0000","last_undegraded":"2024-09-22T03:13:05.257056+0000","last_fullsized":"2024-09-22T03:13:05.257056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44302336,"num_objects":676,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1272,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":54,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:13:05.055218+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:05.055218+0000","last_peered":"2024-09-22T03:13:05.055218+0000","last_clean":"2024-09-22T03:13:05.055218+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:05.055218+0000","last_undegraded":"2024-09-22T03:13:05.055218+0000","last_fullsized":"2024-09-22T03:13:05.055218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":720764952,"num_objects":10999,"num_object_clones":0,"num_object_copies":32997,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20153,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2194,"num_bytes_recovered":143785984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120209408,"data_stored":1923153992,"data_compressed":91321637,"data_compressed_allocated":120197120,"data_compressed_original":1923153920,"omap_allocated":0,"internal_metadata":0},"log_size":20140,"ondisk_log_size":20140,"up":48,"acting":39,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100884,"kb_used_data":39364,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94270956,"statfs":{"total":96636764160,"available":96533458944,"internally_reserved":0,"allocated":40308736,"data_stored":640643657,"data_compressed":30416687,"data_compressed_allocated":40034304,"data_compressed_original":640548864,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79956,"kb_used_data":30916,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94291884,"statfs":{"total":96636764160,"available":96554889216,"internally_reserved":0,"allocated":31657984,"data_stored":501904465,"data_compressed":23830547,"data_compressed_allocated":31367168,"data_compressed_original":501792768,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149731,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72320,"kb_used_data":18480,"kb_used_omap":0,"kb_used_meta":53824,"kb_avail":94299520,"statfs":{"total":96636764160,"available":96562708480,"internally_reserved":0,"allocated":18923520,"data_stored":297705015,"data_compressed":14133555,"data_compressed_allocated":18604032,"data_compressed_original":297582592,"omap_allocated":0,"internal_metadata":55115776},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76140,"kb_used_data":31260,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94295700,"statfs":{"total":96636764160,"available":96558796800,"internally_reserved":0,"allocated":32010240,"data_stored":507540566,"data_compressed":24098180,"data_compressed_allocated":31719424,"data_compressed_original":507428864,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31199232,"data_stored":499122200,"data_compressed":23700992,"data_compressed_allocated":31195136,"data_compressed_original":499122176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18571264,"data_stored":297074712,"data_compressed":14106695,"data_compressed_allocated":18567168,"data_compressed_original":297074688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30965760,"data_stored":495452160,"data_compressed":23526719,"data_compressed_allocated":30965760,"data_compressed_original":495452160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":39473152,"data_stored":631504920,"data_compressed":29987231,"data_compressed_allocated":39469056,"data_compressed_original":631504896,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:08.644 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:08.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:08.794+0000 7f9e7506c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:08.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:08.802+0000 7f9e7506c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:11.042 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:11.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:11.176+0000 7feedacfa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:11.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:11.184+0000 7feed9a98640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:11.263 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:11.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:11.411+0000 7fe4b6a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:11.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:11.420+0000 7fe4b6a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:11.778 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:11.779 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:11.799 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-09-22T03:13:11.249083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654377547,"num_objects":9981,"num_object_clones":0,"num_object_copies":29943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":21314,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":193331200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21261,"ondisk_log_size":21261,"up":75,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":299948,"kb_used_data":111464,"kb_used_omap":0,"kb_used_meta":188416,"kb_avail":377187412,"statfs":{"total":386547056640,"available":386239909888,"internally_reserved":0,"allocated":114139136,"data_stored":1805954505,"data_compressed":85741489,"data_compressed_allocated":112857088,"data_compressed_original":1805467648,"omap_allocated":0,"internal_metadata":192937984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-69861376,"num_objects":-1066,"num_object_clones":0,"num_object_copies":-3198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1066,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":52166656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.867022"},"pg_stats":[{"pgid":"3.f","version":"36'1363","reported_seq":1647,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.220543+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:07.220543+0000","last_peered":"2024-09-22T03:13:07.220543+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:07.220543+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1363,"log_dups_size":0,"ondisk_log_size":1363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44564480,"num_objects":680,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1364,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7471104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"36'1336","reported_seq":1883,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.232946+0000","last_change":"2024-09-22T03:13:02.541383+0000","last_active":"2024-09-22T03:13:08.232946+0000","last_peered":"2024-09-22T03:13:08.232946+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:13:08.232946+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1336,"log_dups_size":0,"ondisk_log_size":1336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41091072,"num_objects":627,"num_object_clones":0,"num_object_copies":1881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1337,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":16056320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":336}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"36'1304","reported_seq":1550,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.212742+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:07.212742+0000","last_peered":"2024-09-22T03:13:07.212742+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:07.212742+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1304,"log_dups_size":0,"ondisk_log_size":1304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632344,"num_objects":621,"num_object_clones":0,"num_object_copies":1863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1305,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":6356992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"36'1326","reported_seq":1396,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:07.145312+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:07.145312+0000","last_peered":"2024-09-22T03:13:07.145312+0000","last_clean":"2024-09-22T03:13:07.145312+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:07.145312+0000","last_undegraded":"2024-09-22T03:13:07.145312+0000","last_fullsized":"2024-09-22T03:13:07.145312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1326,"log_dups_size":0,"ondisk_log_size":1326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43974656,"num_objects":671,"num_object_clones":0,"num_object_copies":2013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"36'1343","reported_seq":1413,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:08.193793+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:08.193793+0000","last_peered":"2024-09-22T03:13:08.193793+0000","last_clean":"2024-09-22T03:13:08.193793+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:08.193793+0000","last_undegraded":"2024-09-22T03:13:08.193793+0000","last_fullsized":"2024-09-22T03:13:08.193793+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1344,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"36'1282","reported_seq":2344,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:07.131672+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:07.131672+0000","last_peered":"2024-09-22T03:13:07.131672+0000","last_clean":"2024-09-22T03:13:07.131672+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:07.131672+0000","last_undegraded":"2024-09-22T03:13:07.131672+0000","last_fullsized":"2024-09-22T03:13:07.131672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270720,"num_objects":645,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1283,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"36'1337","reported_seq":1640,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.211025+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:07.211025+0000","last_peered":"2024-09-22T03:13:07.211025+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:07.211025+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1337,"log_dups_size":0,"ondisk_log_size":1337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535168,"num_objects":588,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1338,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7995392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"36'1383","reported_seq":2454,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:10.227112+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:10.227112+0000","last_peered":"2024-09-22T03:13:10.227112+0000","last_clean":"2024-09-22T03:13:10.227112+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:10.227112+0000","last_undegraded":"2024-09-22T03:13:10.227112+0000","last_fullsized":"2024-09-22T03:13:10.227112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1383,"log_dups_size":0,"ondisk_log_size":1383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":634,"num_object_clones":0,"num_object_copies":1902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1384,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.034097+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:06.034097+0000","last_peered":"2024-09-22T03:13:06.034097+0000","last_clean":"2024-09-22T03:13:06.034097+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:06.034097+0000","last_undegraded":"2024-09-22T03:13:06.034097+0000","last_fullsized":"2024-09-22T03:13:06.034097+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"36'1358","reported_seq":1703,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.229537+0000","last_change":"2024-09-22T03:13:04.661086+0000","last_active":"2024-09-22T03:13:07.229537+0000","last_peered":"2024-09-22T03:13:07.229537+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:07.229537+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41418752,"num_objects":632,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1358,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":9633792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.022028+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:06.022028+0000","last_peered":"2024-09-22T03:13:06.022028+0000","last_clean":"2024-09-22T03:13:06.022028+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:06.022028+0000","last_undegraded":"2024-09-22T03:13:06.022028+0000","last_fullsized":"2024-09-22T03:13:06.022028+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"36'1289","reported_seq":1359,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:08.109598+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:08.109598+0000","last_peered":"2024-09-22T03:13:08.109598+0000","last_clean":"2024-09-22T03:13:08.109598+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:08.109598+0000","last_undegraded":"2024-09-22T03:13:08.109598+0000","last_fullsized":"2024-09-22T03:13:08.109598+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1289,"log_dups_size":0,"ondisk_log_size":1289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40108032,"num_objects":612,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1290,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":91,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025156+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:06.025156+0000","last_peered":"2024-09-22T03:13:06.025156+0000","last_clean":"2024-09-22T03:13:06.025156+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:06.025156+0000","last_undegraded":"2024-09-22T03:13:06.025156+0000","last_fullsized":"2024-09-22T03:13:06.025156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"36'1345","reported_seq":1672,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.200110+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:13:08.200110+0000","last_peered":"2024-09-22T03:13:08.200110+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:08.200110+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1345,"log_dups_size":0,"ondisk_log_size":1345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39976960,"num_objects":610,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":8781824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":251}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.042670+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:06.042670+0000","last_peered":"2024-09-22T03:13:06.042670+0000","last_clean":"2024-09-22T03:13:06.042670+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:06.042670+0000","last_undegraded":"2024-09-22T03:13:06.042670+0000","last_fullsized":"2024-09-22T03:13:06.042670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":213,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025788+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:06.025788+0000","last_peered":"2024-09-22T03:13:06.025788+0000","last_clean":"2024-09-22T03:13:06.025788+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:06.025788+0000","last_undegraded":"2024-09-22T03:13:06.025788+0000","last_fullsized":"2024-09-22T03:13:06.025788+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"36'1313","reported_seq":1625,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.189088+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:08.189088+0000","last_peered":"2024-09-22T03:13:08.189088+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:08.189088+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1313,"log_dups_size":0,"ondisk_log_size":1313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":606,"num_object_clones":0,"num_object_copies":1818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1314,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"36'1283","reported_seq":1558,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.253666+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:13:08.253666+0000","last_peered":"2024-09-22T03:13:08.253666+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:08.253666+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1283,"log_dups_size":0,"ondisk_log_size":1283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41156608,"num_objects":628,"num_object_clones":0,"num_object_copies":1884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1284,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7077888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.036710+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:06.036710+0000","last_peered":"2024-09-22T03:13:06.036710+0000","last_clean":"2024-09-22T03:13:06.036710+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:06.036710+0000","last_undegraded":"2024-09-22T03:13:06.036710+0000","last_fullsized":"2024-09-22T03:13:06.036710+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'1271","reported_seq":2042,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.254802+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:13:08.254802+0000","last_peered":"2024-09-22T03:13:08.254802+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:13:08.254802+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39256064,"num_objects":599,"num_object_clones":0,"num_object_copies":1797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1271,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":23396352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":474}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.027964+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:06.027964+0000","last_peered":"2024-09-22T03:13:06.027964+0000","last_clean":"2024-09-22T03:13:06.027964+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:06.027964+0000","last_undegraded":"2024-09-22T03:13:06.027964+0000","last_fullsized":"2024-09-22T03:13:06.027964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"36'1282","reported_seq":1352,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:08.223582+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:08.223582+0000","last_peered":"2024-09-22T03:13:08.223582+0000","last_clean":"2024-09-22T03:13:08.223582+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:08.223582+0000","last_undegraded":"2024-09-22T03:13:08.223582+0000","last_fullsized":"2024-09-22T03:13:08.223582+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40304640,"num_objects":615,"num_object_clones":0,"num_object_copies":1845,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1283,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.037112+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:06.037112+0000","last_peered":"2024-09-22T03:13:06.037112+0000","last_clean":"2024-09-22T03:13:06.037112+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:06.037112+0000","last_undegraded":"2024-09-22T03:13:06.037112+0000","last_fullsized":"2024-09-22T03:13:06.037112+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"36'1346","reported_seq":2401,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:10.241422+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:10.241422+0000","last_peered":"2024-09-22T03:13:10.241422+0000","last_clean":"2024-09-22T03:13:10.241422+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:10.241422+0000","last_undegraded":"2024-09-22T03:13:10.241422+0000","last_fullsized":"2024-09-22T03:13:10.241422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39387136,"num_objects":601,"num_object_clones":0,"num_object_copies":1803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1347,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.041305+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:06.041305+0000","last_peered":"2024-09-22T03:13:06.041305+0000","last_clean":"2024-09-22T03:13:06.041305+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:06.041305+0000","last_undegraded":"2024-09-22T03:13:06.041305+0000","last_fullsized":"2024-09-22T03:13:06.041305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":653787160,"num_objects":9977,"num_object_clones":0,"num_object_copies":29931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":21175,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":193331200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110821376,"data_stored":1772945480,"data_compressed":84188933,"data_compressed_allocated":110809088,"data_compressed_original":1772945408,"omap_allocated":0,"internal_metadata":0},"log_size":21161,"ondisk_log_size":21161,"up":48,"acting":39,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":80944,"kb_used_data":36064,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94290896,"statfs":{"total":96636764160,"available":96553877504,"internally_reserved":0,"allocated":36929536,"data_stored":586066405,"data_compressed":27824391,"data_compressed_allocated":36622336,"data_compressed_original":585957376,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77408,"kb_used_data":28368,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94294432,"statfs":{"total":96636764160,"available":96557498368,"internally_reserved":0,"allocated":29048832,"data_stored":459647981,"data_compressed":21823307,"data_compressed_allocated":28725248,"data_compressed_original":459522048,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149736,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63804,"kb_used_data":18280,"kb_used_omap":0,"kb_used_meta":45504,"kb_avail":94308036,"statfs":{"total":96636764160,"available":96571428864,"internally_reserved":0,"allocated":18718720,"data_stored":294300677,"data_compressed":13971731,"data_compressed_allocated":18391040,"data_compressed_original":294174720,"omap_allocated":0,"internal_metadata":46596096},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77792,"kb_used_data":28752,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94294048,"statfs":{"total":96636764160,"available":96557105152,"internally_reserved":0,"allocated":29442048,"data_stored":465939442,"data_compressed":22122060,"data_compressed_allocated":29118464,"data_compressed_original":465813504,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28463104,"data_stored":455344152,"data_compressed":21622176,"data_compressed_allocated":28459008,"data_compressed_original":455344128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18341888,"data_stored":293404696,"data_compressed":13932423,"data_compressed_allocated":18337792,"data_compressed_original":293404672,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28119040,"data_stored":449904640,"data_compressed":21363879,"data_compressed_allocated":28119040,"data_compressed_original":449904640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35897344,"data_stored":574291992,"data_compressed":27270455,"data_compressed_allocated":35893248,"data_compressed_original":574291968,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:11.800 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:11.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:11.940+0000 7f8ba24d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:11.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:11.950+0000 7f8ba24d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:12.445 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:12.445 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3024/29943 objects degraded (10.099%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":137,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":36,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":9981,"data_bytes":654377547,"bytes_used":307146752,"bytes_avail":386239909888,"bytes_total":386547056640,"degraded_objects":3024,"degraded_total":29943,"degraded_ratio":0.10099188458070334,"recovering_objects_per_sec":115,"recovering_bytes_per_sec":7596692,"recovering_keys_per_sec":0,"num_objects_recovered":796,"num_bytes_recovered":52166656,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":155},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:12.466 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:12.467 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:12.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:12.619+0000 7f7463b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:12.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:12.630+0000 7f7463b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:12.986 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:12.986 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:13.009 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-09-22T03:13:11.249083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654377547,"num_objects":9981,"num_object_clones":0,"num_object_copies":29943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":21314,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":193331200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21261,"ondisk_log_size":21261,"up":75,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":299948,"kb_used_data":111464,"kb_used_omap":0,"kb_used_meta":188416,"kb_avail":377187412,"statfs":{"total":386547056640,"available":386239909888,"internally_reserved":0,"allocated":114139136,"data_stored":1805954505,"data_compressed":85741489,"data_compressed_allocated":112857088,"data_compressed_original":1805467648,"omap_allocated":0,"internal_metadata":192937984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-69861376,"num_objects":-1066,"num_object_clones":0,"num_object_copies":-3198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1066,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":52166656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.867022"},"pg_stats":[{"pgid":"3.f","version":"36'1363","reported_seq":1647,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.220543+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:07.220543+0000","last_peered":"2024-09-22T03:13:07.220543+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:07.220543+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1363,"log_dups_size":0,"ondisk_log_size":1363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44564480,"num_objects":680,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1364,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7471104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"36'1336","reported_seq":1883,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.232946+0000","last_change":"2024-09-22T03:13:02.541383+0000","last_active":"2024-09-22T03:13:08.232946+0000","last_peered":"2024-09-22T03:13:08.232946+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:13:08.232946+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1336,"log_dups_size":0,"ondisk_log_size":1336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41091072,"num_objects":627,"num_object_clones":0,"num_object_copies":1881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1337,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":16056320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":336}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"36'1304","reported_seq":1550,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.212742+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:07.212742+0000","last_peered":"2024-09-22T03:13:07.212742+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:07.212742+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1304,"log_dups_size":0,"ondisk_log_size":1304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632344,"num_objects":621,"num_object_clones":0,"num_object_copies":1863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1305,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":6356992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"36'1326","reported_seq":1396,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:07.145312+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:07.145312+0000","last_peered":"2024-09-22T03:13:07.145312+0000","last_clean":"2024-09-22T03:13:07.145312+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:07.145312+0000","last_undegraded":"2024-09-22T03:13:07.145312+0000","last_fullsized":"2024-09-22T03:13:07.145312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1326,"log_dups_size":0,"ondisk_log_size":1326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43974656,"num_objects":671,"num_object_clones":0,"num_object_copies":2013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"36'1343","reported_seq":1413,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:08.193793+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:08.193793+0000","last_peered":"2024-09-22T03:13:08.193793+0000","last_clean":"2024-09-22T03:13:08.193793+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:08.193793+0000","last_undegraded":"2024-09-22T03:13:08.193793+0000","last_fullsized":"2024-09-22T03:13:08.193793+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1344,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"36'1282","reported_seq":2344,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:07.131672+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:07.131672+0000","last_peered":"2024-09-22T03:13:07.131672+0000","last_clean":"2024-09-22T03:13:07.131672+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:07.131672+0000","last_undegraded":"2024-09-22T03:13:07.131672+0000","last_fullsized":"2024-09-22T03:13:07.131672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270720,"num_objects":645,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1283,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"36'1337","reported_seq":1640,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.211025+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:07.211025+0000","last_peered":"2024-09-22T03:13:07.211025+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:07.211025+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1337,"log_dups_size":0,"ondisk_log_size":1337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535168,"num_objects":588,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1338,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7995392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"36'1383","reported_seq":2454,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:10.227112+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:10.227112+0000","last_peered":"2024-09-22T03:13:10.227112+0000","last_clean":"2024-09-22T03:13:10.227112+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:10.227112+0000","last_undegraded":"2024-09-22T03:13:10.227112+0000","last_fullsized":"2024-09-22T03:13:10.227112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1383,"log_dups_size":0,"ondisk_log_size":1383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":634,"num_object_clones":0,"num_object_copies":1902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1384,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.034097+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:06.034097+0000","last_peered":"2024-09-22T03:13:06.034097+0000","last_clean":"2024-09-22T03:13:06.034097+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:06.034097+0000","last_undegraded":"2024-09-22T03:13:06.034097+0000","last_fullsized":"2024-09-22T03:13:06.034097+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"36'1358","reported_seq":1703,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:07.229537+0000","last_change":"2024-09-22T03:13:04.661086+0000","last_active":"2024-09-22T03:13:07.229537+0000","last_peered":"2024-09-22T03:13:07.229537+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:07.229537+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41418752,"num_objects":632,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1358,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":9633792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.022028+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:06.022028+0000","last_peered":"2024-09-22T03:13:06.022028+0000","last_clean":"2024-09-22T03:13:06.022028+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:06.022028+0000","last_undegraded":"2024-09-22T03:13:06.022028+0000","last_fullsized":"2024-09-22T03:13:06.022028+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"36'1289","reported_seq":1359,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:08.109598+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:08.109598+0000","last_peered":"2024-09-22T03:13:08.109598+0000","last_clean":"2024-09-22T03:13:08.109598+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:08.109598+0000","last_undegraded":"2024-09-22T03:13:08.109598+0000","last_fullsized":"2024-09-22T03:13:08.109598+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1289,"log_dups_size":0,"ondisk_log_size":1289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40108032,"num_objects":612,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1290,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":91,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025156+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:06.025156+0000","last_peered":"2024-09-22T03:13:06.025156+0000","last_clean":"2024-09-22T03:13:06.025156+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:06.025156+0000","last_undegraded":"2024-09-22T03:13:06.025156+0000","last_fullsized":"2024-09-22T03:13:06.025156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"36'1345","reported_seq":1672,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.200110+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:13:08.200110+0000","last_peered":"2024-09-22T03:13:08.200110+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:08.200110+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1345,"log_dups_size":0,"ondisk_log_size":1345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39976960,"num_objects":610,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":8781824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":251}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.042670+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:06.042670+0000","last_peered":"2024-09-22T03:13:06.042670+0000","last_clean":"2024-09-22T03:13:06.042670+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:06.042670+0000","last_undegraded":"2024-09-22T03:13:06.042670+0000","last_fullsized":"2024-09-22T03:13:06.042670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":213,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025788+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:06.025788+0000","last_peered":"2024-09-22T03:13:06.025788+0000","last_clean":"2024-09-22T03:13:06.025788+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:06.025788+0000","last_undegraded":"2024-09-22T03:13:06.025788+0000","last_fullsized":"2024-09-22T03:13:06.025788+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"36'1313","reported_seq":1625,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.189088+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:08.189088+0000","last_peered":"2024-09-22T03:13:08.189088+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:08.189088+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1313,"log_dups_size":0,"ondisk_log_size":1313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":606,"num_object_clones":0,"num_object_copies":1818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1314,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"36'1283","reported_seq":1558,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.253666+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:13:08.253666+0000","last_peered":"2024-09-22T03:13:08.253666+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:08.253666+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1283,"log_dups_size":0,"ondisk_log_size":1283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41156608,"num_objects":628,"num_object_clones":0,"num_object_copies":1884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1284,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7077888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.036710+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:06.036710+0000","last_peered":"2024-09-22T03:13:06.036710+0000","last_clean":"2024-09-22T03:13:06.036710+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:06.036710+0000","last_undegraded":"2024-09-22T03:13:06.036710+0000","last_fullsized":"2024-09-22T03:13:06.036710+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'1271","reported_seq":2042,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:08.254802+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:13:08.254802+0000","last_peered":"2024-09-22T03:13:08.254802+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:13:08.254802+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39256064,"num_objects":599,"num_object_clones":0,"num_object_copies":1797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1271,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":23396352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":474}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.027964+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:06.027964+0000","last_peered":"2024-09-22T03:13:06.027964+0000","last_clean":"2024-09-22T03:13:06.027964+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:06.027964+0000","last_undegraded":"2024-09-22T03:13:06.027964+0000","last_fullsized":"2024-09-22T03:13:06.027964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"36'1282","reported_seq":1352,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:08.223582+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:08.223582+0000","last_peered":"2024-09-22T03:13:08.223582+0000","last_clean":"2024-09-22T03:13:08.223582+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:08.223582+0000","last_undegraded":"2024-09-22T03:13:08.223582+0000","last_fullsized":"2024-09-22T03:13:08.223582+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40304640,"num_objects":615,"num_object_clones":0,"num_object_copies":1845,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1283,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.037112+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:06.037112+0000","last_peered":"2024-09-22T03:13:06.037112+0000","last_clean":"2024-09-22T03:13:06.037112+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:06.037112+0000","last_undegraded":"2024-09-22T03:13:06.037112+0000","last_fullsized":"2024-09-22T03:13:06.037112+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"36'1346","reported_seq":2401,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:10.241422+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:10.241422+0000","last_peered":"2024-09-22T03:13:10.241422+0000","last_clean":"2024-09-22T03:13:10.241422+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:10.241422+0000","last_undegraded":"2024-09-22T03:13:10.241422+0000","last_fullsized":"2024-09-22T03:13:10.241422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39387136,"num_objects":601,"num_object_clones":0,"num_object_copies":1803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1347,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.041305+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:06.041305+0000","last_peered":"2024-09-22T03:13:06.041305+0000","last_clean":"2024-09-22T03:13:06.041305+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:06.041305+0000","last_undegraded":"2024-09-22T03:13:06.041305+0000","last_fullsized":"2024-09-22T03:13:06.041305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":653787160,"num_objects":9977,"num_object_clones":0,"num_object_copies":29931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":21175,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":193331200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110821376,"data_stored":1772945480,"data_compressed":84188933,"data_compressed_allocated":110809088,"data_compressed_original":1772945408,"omap_allocated":0,"internal_metadata":0},"log_size":21161,"ondisk_log_size":21161,"up":48,"acting":39,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":80944,"kb_used_data":36064,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94290896,"statfs":{"total":96636764160,"available":96553877504,"internally_reserved":0,"allocated":36929536,"data_stored":586066405,"data_compressed":27824391,"data_compressed_allocated":36622336,"data_compressed_original":585957376,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77408,"kb_used_data":28368,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94294432,"statfs":{"total":96636764160,"available":96557498368,"internally_reserved":0,"allocated":29048832,"data_stored":459647981,"data_compressed":21823307,"data_compressed_allocated":28725248,"data_compressed_original":459522048,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149736,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63804,"kb_used_data":18280,"kb_used_omap":0,"kb_used_meta":45504,"kb_avail":94308036,"statfs":{"total":96636764160,"available":96571428864,"internally_reserved":0,"allocated":18718720,"data_stored":294300677,"data_compressed":13971731,"data_compressed_allocated":18391040,"data_compressed_original":294174720,"omap_allocated":0,"internal_metadata":46596096},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77792,"kb_used_data":28752,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94294048,"statfs":{"total":96636764160,"available":96557105152,"internally_reserved":0,"allocated":29442048,"data_stored":465939442,"data_compressed":22122060,"data_compressed_allocated":29118464,"data_compressed_original":465813504,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28463104,"data_stored":455344152,"data_compressed":21622176,"data_compressed_allocated":28459008,"data_compressed_original":455344128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18341888,"data_stored":293404696,"data_compressed":13932423,"data_compressed_allocated":18337792,"data_compressed_original":293404672,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28119040,"data_stored":449904640,"data_compressed":21363879,"data_compressed_allocated":28119040,"data_compressed_original":449904640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35897344,"data_stored":574291992,"data_compressed":27270455,"data_compressed_allocated":35893248,"data_compressed_original":574291968,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:13.433 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:13.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:13.570+0000 7f316824f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:13.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:13.578+0000 7f316824f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:15.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:15.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:15.935+0000 7ff43c04f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:15.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:15.940+0000 7ff43c04f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:16.011 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:16.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:16.161+0000 7f5629997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:16.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:16.173+0000 7f5629997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:16.531 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:16.532 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:16.552 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-22T03:13:16.040938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":581567051,"num_objects":8870,"num_object_clones":0,"num_object_copies":26610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":22425,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3726,"num_bytes_recovered":244187136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22373,"ondisk_log_size":22373,"up":75,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":306108,"kb_used_data":100984,"kb_used_omap":0,"kb_used_meta":205056,"kb_avail":377181252,"statfs":{"total":386547056640,"available":386233602048,"internally_reserved":0,"allocated":103407616,"data_stored":1634122639,"data_compressed":77581825,"data_compressed_allocated":102117376,"data_compressed_original":1633632256,"omap_allocated":0,"internal_metadata":209977344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-139788288,"num_objects":-2133,"num_object_clones":0,"num_object_copies":-6399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2133,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":100401152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.792716"},"pg_stats":[{"pgid":"3.f","version":"36'1446","reported_seq":1896,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.215955+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:12.215955+0000","last_peered":"2024-09-22T03:13:12.215955+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:12.215955+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1446,"log_dups_size":0,"ondisk_log_size":1446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39124992,"num_objects":597,"num_object_clones":0,"num_object_copies":1791,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1447,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":12910592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"36'1405","reported_seq":2233,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.260475+0000","last_change":"2024-09-22T03:13:02.541383+0000","last_active":"2024-09-22T03:13:13.260475+0000","last_peered":"2024-09-22T03:13:13.260475+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:13:13.260475+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1405,"log_dups_size":0,"ondisk_log_size":1405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36634624,"num_objects":559,"num_object_clones":0,"num_object_copies":1677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1405,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":25296896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":409}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"36'1357","reported_seq":1709,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.081094+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:12.081094+0000","last_peered":"2024-09-22T03:13:12.081094+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:12.081094+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1357,"log_dups_size":0,"ondisk_log_size":1357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37158936,"num_objects":568,"num_object_clones":0,"num_object_copies":1704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1358,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"36'1402","reported_seq":1472,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:11.994339+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:11.994339+0000","last_peered":"2024-09-22T03:13:11.994339+0000","last_clean":"2024-09-22T03:13:11.994339+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:11.994339+0000","last_undegraded":"2024-09-22T03:13:11.994339+0000","last_fullsized":"2024-09-22T03:13:11.994339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1402,"log_dups_size":0,"ondisk_log_size":1402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38993920,"num_objects":595,"num_object_clones":0,"num_object_copies":1785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1403,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"36'1405","reported_seq":1475,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:13.123832+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:13.123832+0000","last_peered":"2024-09-22T03:13:13.123832+0000","last_clean":"2024-09-22T03:13:13.123832+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:13.123832+0000","last_undegraded":"2024-09-22T03:13:13.123832+0000","last_fullsized":"2024-09-22T03:13:13.123832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1405,"log_dups_size":0,"ondisk_log_size":1405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":546,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1406,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"36'1361","reported_seq":2423,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:12.164194+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:12.164194+0000","last_peered":"2024-09-22T03:13:12.164194+0000","last_clean":"2024-09-22T03:13:12.164194+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:12.164194+0000","last_undegraded":"2024-09-22T03:13:12.164194+0000","last_fullsized":"2024-09-22T03:13:12.164194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37093376,"num_objects":566,"num_object_clones":0,"num_object_copies":1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"36'1398","reported_seq":1823,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.206954+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:12.206954+0000","last_peered":"2024-09-22T03:13:12.206954+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:12.206954+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1398,"log_dups_size":0,"ondisk_log_size":1398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537472,"num_objects":527,"num_object_clones":0,"num_object_copies":1581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":11993088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"37'1445","reported_seq":2519,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.302807+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:15.302807+0000","last_peered":"2024-09-22T03:13:15.302807+0000","last_clean":"2024-09-22T03:13:15.302807+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:15.302807+0000","last_undegraded":"2024-09-22T03:13:15.302807+0000","last_fullsized":"2024-09-22T03:13:15.302807+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1445,"log_dups_size":0,"ondisk_log_size":1445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37486592,"num_objects":572,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1446,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.034097+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:06.034097+0000","last_peered":"2024-09-22T03:13:06.034097+0000","last_clean":"2024-09-22T03:13:06.034097+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:06.034097+0000","last_undegraded":"2024-09-22T03:13:06.034097+0000","last_fullsized":"2024-09-22T03:13:06.034097+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"36'1434","reported_seq":2035,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.234930+0000","last_change":"2024-09-22T03:13:04.661086+0000","last_active":"2024-09-22T03:13:12.234930+0000","last_peered":"2024-09-22T03:13:12.234930+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:12.234930+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1434,"log_dups_size":0,"ondisk_log_size":1434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36438016,"num_objects":556,"num_object_clones":0,"num_object_copies":1668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":18022400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.022028+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:06.022028+0000","last_peered":"2024-09-22T03:13:06.022028+0000","last_clean":"2024-09-22T03:13:06.022028+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:06.022028+0000","last_undegraded":"2024-09-22T03:13:06.022028+0000","last_fullsized":"2024-09-22T03:13:06.022028+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"36'1361","reported_seq":1431,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:13.041023+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:13.041023+0000","last_peered":"2024-09-22T03:13:13.041023+0000","last_clean":"2024-09-22T03:13:13.041023+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:13.041023+0000","last_undegraded":"2024-09-22T03:13:13.041023+0000","last_fullsized":"2024-09-22T03:13:13.041023+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":91,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025156+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:06.025156+0000","last_peered":"2024-09-22T03:13:06.025156+0000","last_clean":"2024-09-22T03:13:06.025156+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:06.025156+0000","last_undegraded":"2024-09-22T03:13:06.025156+0000","last_fullsized":"2024-09-22T03:13:06.025156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"36'1412","reported_seq":1871,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.221490+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:13:13.221490+0000","last_peered":"2024-09-22T03:13:13.221490+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:13.221490+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35651584,"num_objects":544,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":13107200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":251}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.042670+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:06.042670+0000","last_peered":"2024-09-22T03:13:06.042670+0000","last_clean":"2024-09-22T03:13:06.042670+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:06.042670+0000","last_undegraded":"2024-09-22T03:13:06.042670+0000","last_fullsized":"2024-09-22T03:13:06.042670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":213,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025788+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:06.025788+0000","last_peered":"2024-09-22T03:13:06.025788+0000","last_clean":"2024-09-22T03:13:06.025788+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:06.025788+0000","last_undegraded":"2024-09-22T03:13:06.025788+0000","last_fullsized":"2024-09-22T03:13:06.025788+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"36'1374","reported_seq":1808,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.229611+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:13.229611+0000","last_peered":"2024-09-22T03:13:13.229611+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:13.229611+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1374,"log_dups_size":0,"ondisk_log_size":1374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35717120,"num_objects":545,"num_object_clones":0,"num_object_copies":1635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1375,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":12386304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"36'1367","reported_seq":1810,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.247674+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:13:13.247674+0000","last_peered":"2024-09-22T03:13:13.247674+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:13.247674+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1367,"log_dups_size":0,"ondisk_log_size":1367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35651584,"num_objects":544,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1368,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12582912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.026617+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:15.026617+0000","last_peered":"2024-09-22T03:13:15.026617+0000","last_clean":"2024-09-22T03:13:15.026617+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:15.026617+0000","last_undegraded":"2024-09-22T03:13:15.026617+0000","last_fullsized":"2024-09-22T03:13:15.026617+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'1334","reported_seq":2304,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.241581+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:13:13.241581+0000","last_peered":"2024-09-22T03:13:13.241581+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:13:13.241581+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":535,"num_object_clones":0,"num_object_copies":1605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1335,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":29884416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":509}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.027964+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:06.027964+0000","last_peered":"2024-09-22T03:13:06.027964+0000","last_clean":"2024-09-22T03:13:06.027964+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:06.027964+0000","last_undegraded":"2024-09-22T03:13:06.027964+0000","last_fullsized":"2024-09-22T03:13:06.027964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"36'1361","reported_seq":1431,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:13.089885+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:13.089885+0000","last_peered":"2024-09-22T03:13:13.089885+0000","last_clean":"2024-09-22T03:13:13.089885+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:13.089885+0000","last_undegraded":"2024-09-22T03:13:13.089885+0000","last_fullsized":"2024-09-22T03:13:13.089885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":536,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.026403+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:15.026403+0000","last_peered":"2024-09-22T03:13:15.026403+0000","last_clean":"2024-09-22T03:13:15.026403+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:15.026403+0000","last_undegraded":"2024-09-22T03:13:15.026403+0000","last_fullsized":"2024-09-22T03:13:15.026403+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"37'1411","reported_seq":2469,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.245622+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:15.245622+0000","last_peered":"2024-09-22T03:13:15.245622+0000","last_clean":"2024-09-22T03:13:15.245622+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:15.245622+0000","last_undegraded":"2024-09-22T03:13:15.245622+0000","last_fullsized":"2024-09-22T03:13:15.245622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":536,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.026770+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:15.026770+0000","last_peered":"2024-09-22T03:13:15.026770+0000","last_clean":"2024-09-22T03:13:15.026770+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:15.026770+0000","last_undegraded":"2024-09-22T03:13:15.026770+0000","last_fullsized":"2024-09-22T03:13:15.026770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":580976664,"num_objects":8866,"num_object_clones":0,"num_object_copies":26598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":22286,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3726,"num_bytes_recovered":244187136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100376576,"data_stored":1605828680,"data_compressed":76253333,"data_compressed_allocated":100364288,"data_compressed_original":1605828608,"omap_allocated":0,"internal_metadata":0},"log_size":22273,"ondisk_log_size":22273,"up":48,"acting":39,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81072,"kb_used_data":32032,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94290768,"statfs":{"total":96636764160,"available":96553746432,"internally_reserved":0,"allocated":32800768,"data_stored":520006117,"data_compressed":24687495,"data_compressed_allocated":32493568,"data_compressed_original":519897088,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78384,"kb_used_data":25184,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94293456,"statfs":{"total":96636764160,"available":96556498944,"internally_reserved":0,"allocated":25788416,"data_stored":407481325,"data_compressed":19346155,"data_compressed_allocated":25464832,"data_compressed_original":407355392,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67948,"kb_used_data":18264,"kb_used_omap":0,"kb_used_meta":49664,"kb_avail":94303892,"statfs":{"total":96636764160,"available":96567185408,"internally_reserved":0,"allocated":18702336,"data_stored":293910987,"data_compressed":13953059,"data_compressed_allocated":18366464,"data_compressed_original":293781504,"omap_allocated":0,"internal_metadata":50855936},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78704,"kb_used_data":25504,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94293136,"statfs":{"total":96636764160,"available":96556171264,"internally_reserved":0,"allocated":26116096,"data_stored":412724210,"data_compressed":19595116,"data_compressed_allocated":25792512,"data_compressed_original":412598272,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25255936,"data_stored":404029464,"data_compressed":19185480,"data_compressed_allocated":25251840,"data_compressed_original":404029440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18325504,"data_stored":293142552,"data_compressed":13919975,"data_compressed_allocated":18321408,"data_compressed_original":293142528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24956928,"data_stored":399310848,"data_compressed":18961415,"data_compressed_allocated":24956928,"data_compressed_original":399310848,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31838208,"data_stored":509345816,"data_compressed":24186463,"data_compressed_allocated":31834112,"data_compressed_original":509345792,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:16.554 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:16.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:16.690+0000 7ff2a4797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:16.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:16.698+0000 7ff2a4797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:17.193 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:17.193 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2248/26991 objects degraded (8.329%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":142,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":39,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":8997,"data_bytes":589890123,"bytes_used":309211136,"bytes_avail":386237845504,"bytes_total":386547056640,"degraded_objects":2248,"degraded_total":26991,"degraded_ratio":0.083287021599792524,"recovering_objects_per_sec":165,"recovering_bytes_per_sec":10869977,"recovering_keys_per_sec":0,"num_objects_recovered":1532,"num_bytes_recovered":100401152,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":217},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:17.211 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:17.211 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:17.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:17.361+0000 7f128ccce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:17.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:17.372+0000 7f128ccce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:17.729 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:17.729 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:17.748 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-22T03:13:17.250235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":581567051,"num_objects":8870,"num_object_clones":0,"num_object_copies":26610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":22425,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3726,"num_bytes_recovered":244187136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22373,"ondisk_log_size":22373,"up":75,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":306108,"kb_used_data":100984,"kb_used_omap":0,"kb_used_meta":205056,"kb_avail":377181252,"statfs":{"total":386547056640,"available":386233602048,"internally_reserved":0,"allocated":103407616,"data_stored":1634122639,"data_compressed":77581825,"data_compressed_allocated":102117376,"data_compressed_original":1633632256,"omap_allocated":0,"internal_metadata":209977344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-72810496,"num_objects":-1111,"num_object_clones":0,"num_object_copies":-3333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1111,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":776,"num_bytes_recovered":50855936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001152"},"pg_stats":[{"pgid":"3.f","version":"36'1446","reported_seq":1896,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.215955+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:12.215955+0000","last_peered":"2024-09-22T03:13:12.215955+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:12.215955+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1446,"log_dups_size":0,"ondisk_log_size":1446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39124992,"num_objects":597,"num_object_clones":0,"num_object_copies":1791,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1447,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":12910592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"36'1405","reported_seq":2233,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.260475+0000","last_change":"2024-09-22T03:13:02.541383+0000","last_active":"2024-09-22T03:13:13.260475+0000","last_peered":"2024-09-22T03:13:13.260475+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:12:32.538260+0000","last_became_peered":"2024-09-22T03:12:32.538260+0000","last_unstale":"2024-09-22T03:13:13.260475+0000","last_undegraded":"2024-09-22T03:12:32.071177+0000","last_fullsized":"2024-09-22T03:12:32.050691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1405,"log_dups_size":0,"ondisk_log_size":1405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36634624,"num_objects":559,"num_object_clones":0,"num_object_copies":1677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1405,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":25296896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":409}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"36'1357","reported_seq":1709,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.081094+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:12.081094+0000","last_peered":"2024-09-22T03:13:12.081094+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:12.081094+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1357,"log_dups_size":0,"ondisk_log_size":1357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37158936,"num_objects":568,"num_object_clones":0,"num_object_copies":1704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1358,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"36'1402","reported_seq":1472,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:11.994339+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:11.994339+0000","last_peered":"2024-09-22T03:13:11.994339+0000","last_clean":"2024-09-22T03:13:11.994339+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:11.994339+0000","last_undegraded":"2024-09-22T03:13:11.994339+0000","last_fullsized":"2024-09-22T03:13:11.994339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1402,"log_dups_size":0,"ondisk_log_size":1402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38993920,"num_objects":595,"num_object_clones":0,"num_object_copies":1785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1403,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"36'1405","reported_seq":1475,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:13.123832+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:13.123832+0000","last_peered":"2024-09-22T03:13:13.123832+0000","last_clean":"2024-09-22T03:13:13.123832+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:13.123832+0000","last_undegraded":"2024-09-22T03:13:13.123832+0000","last_fullsized":"2024-09-22T03:13:13.123832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1405,"log_dups_size":0,"ondisk_log_size":1405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":546,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1406,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"36'1361","reported_seq":2423,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:12.164194+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:12.164194+0000","last_peered":"2024-09-22T03:13:12.164194+0000","last_clean":"2024-09-22T03:13:12.164194+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:12.164194+0000","last_undegraded":"2024-09-22T03:13:12.164194+0000","last_fullsized":"2024-09-22T03:13:12.164194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37093376,"num_objects":566,"num_object_clones":0,"num_object_copies":1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"36'1398","reported_seq":1823,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.206954+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:12.206954+0000","last_peered":"2024-09-22T03:13:12.206954+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:12.206954+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1398,"log_dups_size":0,"ondisk_log_size":1398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537472,"num_objects":527,"num_object_clones":0,"num_object_copies":1581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":11993088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"37'1445","reported_seq":2519,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.302807+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:15.302807+0000","last_peered":"2024-09-22T03:13:15.302807+0000","last_clean":"2024-09-22T03:13:15.302807+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:15.302807+0000","last_undegraded":"2024-09-22T03:13:15.302807+0000","last_fullsized":"2024-09-22T03:13:15.302807+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1445,"log_dups_size":0,"ondisk_log_size":1445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37486592,"num_objects":572,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1446,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.034097+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:06.034097+0000","last_peered":"2024-09-22T03:13:06.034097+0000","last_clean":"2024-09-22T03:13:06.034097+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:06.034097+0000","last_undegraded":"2024-09-22T03:13:06.034097+0000","last_fullsized":"2024-09-22T03:13:06.034097+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"36'1434","reported_seq":2035,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:12.234930+0000","last_change":"2024-09-22T03:13:04.661086+0000","last_active":"2024-09-22T03:13:12.234930+0000","last_peered":"2024-09-22T03:13:12.234930+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:12.234930+0000","last_undegraded":"2024-09-22T03:12:32.076673+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1434,"log_dups_size":0,"ondisk_log_size":1434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36438016,"num_objects":556,"num_object_clones":0,"num_object_copies":1668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":18022400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.022028+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:06.022028+0000","last_peered":"2024-09-22T03:13:06.022028+0000","last_clean":"2024-09-22T03:13:06.022028+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:06.022028+0000","last_undegraded":"2024-09-22T03:13:06.022028+0000","last_fullsized":"2024-09-22T03:13:06.022028+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"36'1361","reported_seq":1431,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:13.041023+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:13.041023+0000","last_peered":"2024-09-22T03:13:13.041023+0000","last_clean":"2024-09-22T03:13:13.041023+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:13.041023+0000","last_undegraded":"2024-09-22T03:13:13.041023+0000","last_fullsized":"2024-09-22T03:13:13.041023+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":91,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025156+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:06.025156+0000","last_peered":"2024-09-22T03:13:06.025156+0000","last_clean":"2024-09-22T03:13:06.025156+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:06.025156+0000","last_undegraded":"2024-09-22T03:13:06.025156+0000","last_fullsized":"2024-09-22T03:13:06.025156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"36'1412","reported_seq":1871,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.221490+0000","last_change":"2024-09-22T03:12:32.595169+0000","last_active":"2024-09-22T03:13:13.221490+0000","last_peered":"2024-09-22T03:13:13.221490+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:13.221490+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35651584,"num_objects":544,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":13107200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":251}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.042670+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:06.042670+0000","last_peered":"2024-09-22T03:13:06.042670+0000","last_clean":"2024-09-22T03:13:06.042670+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:06.042670+0000","last_undegraded":"2024-09-22T03:13:06.042670+0000","last_fullsized":"2024-09-22T03:13:06.042670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":213,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.025788+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:06.025788+0000","last_peered":"2024-09-22T03:13:06.025788+0000","last_clean":"2024-09-22T03:13:06.025788+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:06.025788+0000","last_undegraded":"2024-09-22T03:13:06.025788+0000","last_fullsized":"2024-09-22T03:13:06.025788+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"36'1374","reported_seq":1808,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.229611+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:13.229611+0000","last_peered":"2024-09-22T03:13:13.229611+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:13.229611+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1374,"log_dups_size":0,"ondisk_log_size":1374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35717120,"num_objects":545,"num_object_clones":0,"num_object_copies":1635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1375,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":12386304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"36'1367","reported_seq":1810,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.247674+0000","last_change":"2024-09-22T03:12:32.618584+0000","last_active":"2024-09-22T03:13:13.247674+0000","last_peered":"2024-09-22T03:13:13.247674+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:13.247674+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1367,"log_dups_size":0,"ondisk_log_size":1367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35651584,"num_objects":544,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1368,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":12582912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.026617+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:15.026617+0000","last_peered":"2024-09-22T03:13:15.026617+0000","last_clean":"2024-09-22T03:13:15.026617+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:15.026617+0000","last_undegraded":"2024-09-22T03:13:15.026617+0000","last_fullsized":"2024-09-22T03:13:15.026617+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'1334","reported_seq":2304,"reported_epoch":36,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:13.241581+0000","last_change":"2024-09-22T03:12:46.697826+0000","last_active":"2024-09-22T03:13:13.241581+0000","last_peered":"2024-09-22T03:13:13.241581+0000","last_clean":"2024-09-22T03:11:38.299626+0000","last_became_active":"2024-09-22T03:12:32.605031+0000","last_became_peered":"2024-09-22T03:12:32.605031+0000","last_unstale":"2024-09-22T03:13:13.241581+0000","last_undegraded":"2024-09-22T03:12:32.221737+0000","last_fullsized":"2024-09-22T03:12:32.145387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":535,"num_object_clones":0,"num_object_copies":1605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1335,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":29884416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":509}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:06.027964+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:06.027964+0000","last_peered":"2024-09-22T03:13:06.027964+0000","last_clean":"2024-09-22T03:13:06.027964+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:06.027964+0000","last_undegraded":"2024-09-22T03:13:06.027964+0000","last_fullsized":"2024-09-22T03:13:06.027964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"36'1361","reported_seq":1431,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-22T03:13:13.089885+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:13.089885+0000","last_peered":"2024-09-22T03:13:13.089885+0000","last_clean":"2024-09-22T03:13:13.089885+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:13.089885+0000","last_undegraded":"2024-09-22T03:13:13.089885+0000","last_fullsized":"2024-09-22T03:13:13.089885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":536,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.026403+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:15.026403+0000","last_peered":"2024-09-22T03:13:15.026403+0000","last_clean":"2024-09-22T03:13:15.026403+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:15.026403+0000","last_undegraded":"2024-09-22T03:13:15.026403+0000","last_fullsized":"2024-09-22T03:13:15.026403+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"37'1411","reported_seq":2469,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.245622+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:15.245622+0000","last_peered":"2024-09-22T03:13:15.245622+0000","last_clean":"2024-09-22T03:13:15.245622+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:15.245622+0000","last_undegraded":"2024-09-22T03:13:15.245622+0000","last_fullsized":"2024-09-22T03:13:15.245622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":536,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:13:15.026770+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:15.026770+0000","last_peered":"2024-09-22T03:13:15.026770+0000","last_clean":"2024-09-22T03:13:15.026770+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:15.026770+0000","last_undegraded":"2024-09-22T03:13:15.026770+0000","last_fullsized":"2024-09-22T03:13:15.026770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":580976664,"num_objects":8866,"num_object_clones":0,"num_object_copies":26598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":22286,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3726,"num_bytes_recovered":244187136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100376576,"data_stored":1605828680,"data_compressed":76253333,"data_compressed_allocated":100364288,"data_compressed_original":1605828608,"omap_allocated":0,"internal_metadata":0},"log_size":22273,"ondisk_log_size":22273,"up":48,"acting":39,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81072,"kb_used_data":32032,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94290768,"statfs":{"total":96636764160,"available":96553746432,"internally_reserved":0,"allocated":32800768,"data_stored":520006117,"data_compressed":24687495,"data_compressed_allocated":32493568,"data_compressed_original":519897088,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78384,"kb_used_data":25184,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94293456,"statfs":{"total":96636764160,"available":96556498944,"internally_reserved":0,"allocated":25788416,"data_stored":407481325,"data_compressed":19346155,"data_compressed_allocated":25464832,"data_compressed_original":407355392,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67948,"kb_used_data":18264,"kb_used_omap":0,"kb_used_meta":49664,"kb_avail":94303892,"statfs":{"total":96636764160,"available":96567185408,"internally_reserved":0,"allocated":18702336,"data_stored":293910987,"data_compressed":13953059,"data_compressed_allocated":18366464,"data_compressed_original":293781504,"omap_allocated":0,"internal_metadata":50855936},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78704,"kb_used_data":25504,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94293136,"statfs":{"total":96636764160,"available":96556171264,"internally_reserved":0,"allocated":26116096,"data_stored":412724210,"data_compressed":19595116,"data_compressed_allocated":25792512,"data_compressed_original":412598272,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25255936,"data_stored":404029464,"data_compressed":19185480,"data_compressed_allocated":25251840,"data_compressed_original":404029440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18325504,"data_stored":293142552,"data_compressed":13919975,"data_compressed_allocated":18321408,"data_compressed_original":293142528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24956928,"data_stored":399310848,"data_compressed":18961415,"data_compressed_allocated":24956928,"data_compressed_original":399310848,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31838208,"data_stored":509345816,"data_compressed":24186463,"data_compressed_allocated":31834112,"data_compressed_original":509345792,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:18.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:18.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:18.177+0000 7f2ba1863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:18.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:18.186+0000 7f2ba1863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:20.422 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:20.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:20.554+0000 7f5c92117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:20.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:20.560+0000 7f5c92117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:20.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:20.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:20.875+0000 7fbdb9c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:20.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:20.884+0000 7fbdb9c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:21.222 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:21.223 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:21.241 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-22T03:13:21.008814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519635531,"num_objects":7925,"num_object_clones":0,"num_object_copies":23775,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":23370,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":300089344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23319,"ondisk_log_size":23319,"up":75,"acting":69,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":310380,"kb_used_data":92768,"kb_used_omap":0,"kb_used_meta":217536,"kb_avail":377176980,"statfs":{"total":386547056640,"available":386229227520,"internally_reserved":0,"allocated":94994432,"data_stored":1497980864,"data_compressed":71115089,"data_compressed_allocated":93605888,"data_compressed_original":1497448448,"omap_allocated":0,"internal_metadata":222756864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-61931520,"num_objects":-945,"num_object_clones":0,"num_object_copies":-2835,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":55902208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.967875"},"pg_stats":[{"pgid":"3.f","version":"39'1509","reported_seq":2091,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:17.146609+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:17.146609+0000","last_peered":"2024-09-22T03:13:17.146609+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:17.146609+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1509,"log_dups_size":0,"ondisk_log_size":1509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34996224,"num_objects":534,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":17039360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"40'1455","reported_seq":2601,"reported_epoch":40,"state":"active+recovery_wait","last_fresh":"2024-09-22T03:13:18.243344+0000","last_change":"2024-09-22T03:13:18.131280+0000","last_active":"2024-09-22T03:13:18.243344+0000","last_peered":"2024-09-22T03:13:18.243344+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:18.243344+0000","last_undegraded":"2024-09-22T03:13:18.243344+0000","last_fullsized":"2024-09-22T03:13:18.243344+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1455,"log_dups_size":0,"ondisk_log_size":1455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33292288,"num_objects":508,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1456,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":["2","1","3"],"object_location_counts":[{"shards":"1,2,3","objects":508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"38'1423","reported_seq":1912,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:17.238994+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:17.238994+0000","last_peered":"2024-09-22T03:13:17.238994+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:17.238994+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1423,"log_dups_size":0,"ondisk_log_size":1423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899096,"num_objects":503,"num_object_clones":0,"num_object_copies":1509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1423,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":14155776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"39'1481","reported_seq":1560,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.074068+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:17.074068+0000","last_peered":"2024-09-22T03:13:17.074068+0000","last_clean":"2024-09-22T03:13:17.074068+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:17.074068+0000","last_undegraded":"2024-09-22T03:13:17.074068+0000","last_fullsized":"2024-09-22T03:13:17.074068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33816576,"num_objects":516,"num_object_clones":0,"num_object_copies":1548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1482,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"40'1464","reported_seq":1546,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.191959+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:18.191959+0000","last_peered":"2024-09-22T03:13:18.191959+0000","last_clean":"2024-09-22T03:13:18.191959+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:18.191959+0000","last_undegraded":"2024-09-22T03:13:18.191959+0000","last_fullsized":"2024-09-22T03:13:18.191959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1464,"log_dups_size":0,"ondisk_log_size":1464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":487,"num_object_clones":0,"num_object_copies":1461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"39'1430","reported_seq":2501,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.222595+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:17.222595+0000","last_peered":"2024-09-22T03:13:17.222595+0000","last_clean":"2024-09-22T03:13:17.222595+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:17.222595+0000","last_undegraded":"2024-09-22T03:13:17.222595+0000","last_fullsized":"2024-09-22T03:13:17.222595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1430,"log_dups_size":0,"ondisk_log_size":1430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":497,"num_object_clones":0,"num_object_copies":1491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"39'1450","reported_seq":1985,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:17.208448+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:17.208448+0000","last_peered":"2024-09-22T03:13:17.208448+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:17.208448+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1450,"log_dups_size":0,"ondisk_log_size":1450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":475,"num_object_clones":0,"num_object_copies":1425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1451,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15400960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"40'1505","reported_seq":2590,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039583+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:20.039583+0000","last_peered":"2024-09-22T03:13:20.039583+0000","last_clean":"2024-09-22T03:13:20.039583+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:20.039583+0000","last_undegraded":"2024-09-22T03:13:20.039583+0000","last_fullsized":"2024-09-22T03:13:20.039583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1505,"log_dups_size":0,"ondisk_log_size":1505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":513,"num_object_clones":0,"num_object_copies":1539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1505,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.058246+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:18.058246+0000","last_peered":"2024-09-22T03:13:18.058246+0000","last_clean":"2024-09-22T03:13:18.058246+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:18.058246+0000","last_undegraded":"2024-09-22T03:13:18.058246+0000","last_fullsized":"2024-09-22T03:13:18.058246+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"40'1513","reported_seq":2578,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-22T03:13:20.036139+0000","last_change":"2024-09-22T03:13:20.113262+0000","last_active":"2024-09-22T03:13:20.113262+0000","last_peered":"2024-09-22T03:13:20.036139+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:20.036139+0000","last_undegraded":"2024-09-22T03:13:20.036139+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1513,"log_dups_size":0,"ondisk_log_size":1513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1513,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.058309+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:17.058309+0000","last_peered":"2024-09-22T03:13:17.058309+0000","last_clean":"2024-09-22T03:13:17.058309+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:17.058309+0000","last_undegraded":"2024-09-22T03:13:17.058309+0000","last_fullsized":"2024-09-22T03:13:17.058309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'1417","reported_seq":1499,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.132921+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:18.132921+0000","last_peered":"2024-09-22T03:13:18.132921+0000","last_clean":"2024-09-22T03:13:18.132921+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:18.132921+0000","last_undegraded":"2024-09-22T03:13:18.132921+0000","last_fullsized":"2024-09-22T03:13:18.132921+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1417,"log_dups_size":0,"ondisk_log_size":1417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1418,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.059464+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:18.059464+0000","last_peered":"2024-09-22T03:13:18.059464+0000","last_clean":"2024-09-22T03:13:18.059464+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:18.059464+0000","last_undegraded":"2024-09-22T03:13:18.059464+0000","last_fullsized":"2024-09-22T03:13:18.059464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'1479","reported_seq":2247,"reported_epoch":40,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:18.226011+0000","last_change":"2024-09-22T03:13:14.287762+0000","last_active":"2024-09-22T03:13:18.226011+0000","last_peered":"2024-09-22T03:13:18.226011+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:18.226011+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1479,"log_dups_size":0,"ondisk_log_size":1479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":476,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":22937600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":333}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.063206+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:18.063206+0000","last_peered":"2024-09-22T03:13:18.063206+0000","last_clean":"2024-09-22T03:13:18.063206+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:18.063206+0000","last_undegraded":"2024-09-22T03:13:18.063206+0000","last_fullsized":"2024-09-22T03:13:18.063206+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":222,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.060295+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:17.060295+0000","last_peered":"2024-09-22T03:13:17.060295+0000","last_clean":"2024-09-22T03:13:17.060295+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:17.060295+0000","last_undegraded":"2024-09-22T03:13:17.060295+0000","last_fullsized":"2024-09-22T03:13:17.060295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"39'1419","reported_seq":1951,"reported_epoch":40,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:18.203682+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:18.203682+0000","last_peered":"2024-09-22T03:13:18.203682+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:18.203682+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1419,"log_dups_size":0,"ondisk_log_size":1419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768000,"num_objects":500,"num_object_clones":0,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1420,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":15335424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'1407","reported_seq":2008,"reported_epoch":40,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:18.264300+0000","last_change":"2024-09-22T03:13:16.793045+0000","last_active":"2024-09-22T03:13:18.264300+0000","last_peered":"2024-09-22T03:13:18.264300+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:18.264300+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1407,"log_dups_size":0,"ondisk_log_size":1407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":505,"num_object_clones":0,"num_object_copies":1515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1407,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":17498112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.041961+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:20.041961+0000","last_peered":"2024-09-22T03:13:20.041961+0000","last_clean":"2024-09-22T03:13:20.041961+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:20.041961+0000","last_undegraded":"2024-09-22T03:13:20.041961+0000","last_fullsized":"2024-09-22T03:13:20.041961+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"40'1393","reported_seq":2431,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.180514+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:18.180514+0000","last_peered":"2024-09-22T03:13:18.180514+0000","last_clean":"2024-09-22T03:13:18.180514+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:18.180514+0000","last_undegraded":"2024-09-22T03:13:18.180514+0000","last_fullsized":"2024-09-22T03:13:18.180514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1393,"log_dups_size":0,"ondisk_log_size":1393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":476,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1394,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.062522+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:18.062522+0000","last_peered":"2024-09-22T03:13:18.062522+0000","last_clean":"2024-09-22T03:13:18.062522+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:18.062522+0000","last_undegraded":"2024-09-22T03:13:18.062522+0000","last_fullsized":"2024-09-22T03:13:18.062522+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"40'1411","reported_seq":1493,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.185646+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:18.185646+0000","last_peered":"2024-09-22T03:13:18.185646+0000","last_clean":"2024-09-22T03:13:18.185646+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:18.185646+0000","last_undegraded":"2024-09-22T03:13:18.185646+0000","last_fullsized":"2024-09-22T03:13:18.185646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039218+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:20.039218+0000","last_peered":"2024-09-22T03:13:20.039218+0000","last_clean":"2024-09-22T03:13:20.039218+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:20.039218+0000","last_undegraded":"2024-09-22T03:13:20.039218+0000","last_fullsized":"2024-09-22T03:13:20.039218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"41'1463","reported_seq":2533,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039459+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:20.039459+0000","last_peered":"2024-09-22T03:13:20.039459+0000","last_clean":"2024-09-22T03:13:20.039459+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:20.039459+0000","last_undegraded":"2024-09-22T03:13:20.039459+0000","last_fullsized":"2024-09-22T03:13:20.039459+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1463,"log_dups_size":0,"ondisk_log_size":1463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1464,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.040632+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:20.040632+0000","last_peered":"2024-09-22T03:13:20.040632+0000","last_clean":"2024-09-22T03:13:20.040632+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:20.040632+0000","last_undegraded":"2024-09-22T03:13:20.040632+0000","last_fullsized":"2024-09-22T03:13:20.040632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":519045144,"num_objects":7921,"num_object_clones":0,"num_object_copies":23763,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":23231,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":300089344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":92045312,"data_stored":1472528456,"data_compressed":69923525,"data_compressed_allocated":92033024,"data_compressed_original":1472528384,"omap_allocated":0,"internal_metadata":0},"log_size":23219,"ondisk_log_size":23219,"up":48,"acting":42,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673094,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82024,"kb_used_data":28820,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94289816,"statfs":{"total":96636764160,"available":96552771584,"internally_reserved":0,"allocated":29511680,"data_stored":466998021,"data_compressed":22169887,"data_compressed_allocated":29179904,"data_compressed_original":466878464,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673094,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79976,"kb_used_data":22616,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94291864,"statfs":{"total":96636764160,"available":96554868736,"internally_reserved":0,"allocated":23158784,"data_stored":365024525,"data_compressed":17329579,"data_compressed_allocated":22810624,"data_compressed_original":364888064,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149746,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72204,"kb_used_data":18360,"kb_used_omap":0,"kb_used_meta":53824,"kb_avail":94299636,"statfs":{"total":96636764160,"available":96562827264,"internally_reserved":0,"allocated":18800640,"data_stored":294973509,"data_compressed":14002851,"data_compressed_allocated":18432000,"data_compressed_original":294830080,"omap_allocated":0,"internal_metadata":55115776},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76176,"kb_used_data":22972,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94295664,"statfs":{"total":96636764160,"available":96558759936,"internally_reserved":0,"allocated":23523328,"data_stored":370984809,"data_compressed":17612772,"data_compressed_allocated":23183360,"data_compressed_original":370851840,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22585344,"data_stored":361299992,"data_compressed":17156456,"data_compressed_allocated":22581248,"data_compressed_original":361299968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18444288,"data_stored":295043096,"data_compressed":14010223,"data_compressed_allocated":18440192,"data_compressed_original":295043072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22355968,"data_stored":357695488,"data_compressed":16985295,"data_compressed_allocated":22355968,"data_compressed_original":357695488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28659712,"data_stored":458489880,"data_compressed":21771551,"data_compressed_allocated":28655616,"data_compressed_original":458489856,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:21.242 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:21.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:21.387+0000 7f60cef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:21.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:21.397+0000 7f60cef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:21.808 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:21.808 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1474/24204 objects degraded (6.090%), 7 pgs degraded","count":7},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":146,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":42,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":3},{"state_name":"active+recovery_wait","count":1}],"num_pgs":25,"num_pools":3,"num_objects":8068,"data_bytes":529007179,"bytes_used":313470976,"bytes_avail":386233585664,"bytes_total":386547056640,"degraded_objects":1474,"degraded_total":24204,"degraded_ratio":0.060899024954552967,"recovering_objects_per_sec":182,"recovering_bytes_per_sec":11954413,"recovering_keys_per_sec":0,"num_objects_recovered":774,"num_bytes_recovered":50724864,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":218},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:21.827 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:21.827 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:21.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:21.970+0000 7fe456223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:21.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:21.979+0000 7fe456223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:22.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:22.272 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:22.290 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-22T03:13:21.250926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519635531,"num_objects":7925,"num_object_clones":0,"num_object_copies":23775,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":23370,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":300089344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23319,"ondisk_log_size":23319,"up":75,"acting":69,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":310380,"kb_used_data":92768,"kb_used_omap":0,"kb_used_meta":217536,"kb_avail":377176980,"statfs":{"total":386547056640,"available":386229227520,"internally_reserved":0,"allocated":94994432,"data_stored":1497980864,"data_compressed":71115089,"data_compressed_allocated":93605888,"data_compressed_original":1497448448,"omap_allocated":0,"internal_metadata":222756864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-61931520,"num_objects":-945,"num_object_clones":0,"num_object_copies":-2835,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":55902208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.206527"},"pg_stats":[{"pgid":"3.f","version":"39'1509","reported_seq":2091,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:17.146609+0000","last_change":"2024-09-22T03:12:32.591067+0000","last_active":"2024-09-22T03:13:17.146609+0000","last_peered":"2024-09-22T03:13:17.146609+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:17.146609+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1509,"log_dups_size":0,"ondisk_log_size":1509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34996224,"num_objects":534,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":17039360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"40'1455","reported_seq":2601,"reported_epoch":40,"state":"active+recovery_wait","last_fresh":"2024-09-22T03:13:18.243344+0000","last_change":"2024-09-22T03:13:18.131280+0000","last_active":"2024-09-22T03:13:18.243344+0000","last_peered":"2024-09-22T03:13:18.243344+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:18.243344+0000","last_undegraded":"2024-09-22T03:13:18.243344+0000","last_fullsized":"2024-09-22T03:13:18.243344+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1455,"log_dups_size":0,"ondisk_log_size":1455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33292288,"num_objects":508,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1456,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":["2","1","3"],"object_location_counts":[{"shards":"1,2,3","objects":508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"38'1423","reported_seq":1912,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:17.238994+0000","last_change":"2024-09-22T03:12:32.558575+0000","last_active":"2024-09-22T03:13:17.238994+0000","last_peered":"2024-09-22T03:13:17.238994+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:17.238994+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1423,"log_dups_size":0,"ondisk_log_size":1423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899096,"num_objects":503,"num_object_clones":0,"num_object_copies":1509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1423,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":14155776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"39'1481","reported_seq":1560,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.074068+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:17.074068+0000","last_peered":"2024-09-22T03:13:17.074068+0000","last_clean":"2024-09-22T03:13:17.074068+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:17.074068+0000","last_undegraded":"2024-09-22T03:13:17.074068+0000","last_fullsized":"2024-09-22T03:13:17.074068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33816576,"num_objects":516,"num_object_clones":0,"num_object_copies":1548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1482,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"40'1464","reported_seq":1546,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.191959+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:18.191959+0000","last_peered":"2024-09-22T03:13:18.191959+0000","last_clean":"2024-09-22T03:13:18.191959+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:18.191959+0000","last_undegraded":"2024-09-22T03:13:18.191959+0000","last_fullsized":"2024-09-22T03:13:18.191959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1464,"log_dups_size":0,"ondisk_log_size":1464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":487,"num_object_clones":0,"num_object_copies":1461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"39'1430","reported_seq":2501,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.222595+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:17.222595+0000","last_peered":"2024-09-22T03:13:17.222595+0000","last_clean":"2024-09-22T03:13:17.222595+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:17.222595+0000","last_undegraded":"2024-09-22T03:13:17.222595+0000","last_fullsized":"2024-09-22T03:13:17.222595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1430,"log_dups_size":0,"ondisk_log_size":1430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":497,"num_object_clones":0,"num_object_copies":1491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"39'1450","reported_seq":1985,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:17.208448+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:17.208448+0000","last_peered":"2024-09-22T03:13:17.208448+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:17.208448+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1450,"log_dups_size":0,"ondisk_log_size":1450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":475,"num_object_clones":0,"num_object_copies":1425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1451,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15400960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"40'1505","reported_seq":2590,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039583+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:20.039583+0000","last_peered":"2024-09-22T03:13:20.039583+0000","last_clean":"2024-09-22T03:13:20.039583+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:20.039583+0000","last_undegraded":"2024-09-22T03:13:20.039583+0000","last_fullsized":"2024-09-22T03:13:20.039583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1505,"log_dups_size":0,"ondisk_log_size":1505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":513,"num_object_clones":0,"num_object_copies":1539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1505,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.058246+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:18.058246+0000","last_peered":"2024-09-22T03:13:18.058246+0000","last_clean":"2024-09-22T03:13:18.058246+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:18.058246+0000","last_undegraded":"2024-09-22T03:13:18.058246+0000","last_fullsized":"2024-09-22T03:13:18.058246+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"40'1513","reported_seq":2578,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-22T03:13:20.036139+0000","last_change":"2024-09-22T03:13:20.113262+0000","last_active":"2024-09-22T03:13:20.113262+0000","last_peered":"2024-09-22T03:13:20.036139+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:20.036139+0000","last_undegraded":"2024-09-22T03:13:20.036139+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1513,"log_dups_size":0,"ondisk_log_size":1513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1513,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.058309+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:17.058309+0000","last_peered":"2024-09-22T03:13:17.058309+0000","last_clean":"2024-09-22T03:13:17.058309+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:17.058309+0000","last_undegraded":"2024-09-22T03:13:17.058309+0000","last_fullsized":"2024-09-22T03:13:17.058309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'1417","reported_seq":1499,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.132921+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:18.132921+0000","last_peered":"2024-09-22T03:13:18.132921+0000","last_clean":"2024-09-22T03:13:18.132921+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:18.132921+0000","last_undegraded":"2024-09-22T03:13:18.132921+0000","last_fullsized":"2024-09-22T03:13:18.132921+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1417,"log_dups_size":0,"ondisk_log_size":1417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1418,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.059464+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:18.059464+0000","last_peered":"2024-09-22T03:13:18.059464+0000","last_clean":"2024-09-22T03:13:18.059464+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:18.059464+0000","last_undegraded":"2024-09-22T03:13:18.059464+0000","last_fullsized":"2024-09-22T03:13:18.059464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'1479","reported_seq":2247,"reported_epoch":40,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:18.226011+0000","last_change":"2024-09-22T03:13:14.287762+0000","last_active":"2024-09-22T03:13:18.226011+0000","last_peered":"2024-09-22T03:13:18.226011+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:18.226011+0000","last_undegraded":"2024-09-22T03:12:32.042316+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1479,"log_dups_size":0,"ondisk_log_size":1479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":476,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":22937600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":333}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.063206+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:18.063206+0000","last_peered":"2024-09-22T03:13:18.063206+0000","last_clean":"2024-09-22T03:13:18.063206+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:18.063206+0000","last_undegraded":"2024-09-22T03:13:18.063206+0000","last_fullsized":"2024-09-22T03:13:18.063206+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":222,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:13:17.060295+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:17.060295+0000","last_peered":"2024-09-22T03:13:17.060295+0000","last_clean":"2024-09-22T03:13:17.060295+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:17.060295+0000","last_undegraded":"2024-09-22T03:13:17.060295+0000","last_fullsized":"2024-09-22T03:13:17.060295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"39'1419","reported_seq":1951,"reported_epoch":40,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:18.203682+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:18.203682+0000","last_peered":"2024-09-22T03:13:18.203682+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:18.203682+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1419,"log_dups_size":0,"ondisk_log_size":1419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768000,"num_objects":500,"num_object_clones":0,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1420,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":15335424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'1407","reported_seq":2008,"reported_epoch":40,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:18.264300+0000","last_change":"2024-09-22T03:13:16.793045+0000","last_active":"2024-09-22T03:13:18.264300+0000","last_peered":"2024-09-22T03:13:18.264300+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:18.264300+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1407,"log_dups_size":0,"ondisk_log_size":1407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":505,"num_object_clones":0,"num_object_copies":1515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1407,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":17498112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.041961+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:20.041961+0000","last_peered":"2024-09-22T03:13:20.041961+0000","last_clean":"2024-09-22T03:13:20.041961+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:20.041961+0000","last_undegraded":"2024-09-22T03:13:20.041961+0000","last_fullsized":"2024-09-22T03:13:20.041961+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"40'1393","reported_seq":2431,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.180514+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:18.180514+0000","last_peered":"2024-09-22T03:13:18.180514+0000","last_clean":"2024-09-22T03:13:18.180514+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:18.180514+0000","last_undegraded":"2024-09-22T03:13:18.180514+0000","last_fullsized":"2024-09-22T03:13:18.180514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1393,"log_dups_size":0,"ondisk_log_size":1393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":476,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1394,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.062522+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:18.062522+0000","last_peered":"2024-09-22T03:13:18.062522+0000","last_clean":"2024-09-22T03:13:18.062522+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:18.062522+0000","last_undegraded":"2024-09-22T03:13:18.062522+0000","last_fullsized":"2024-09-22T03:13:18.062522+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"40'1411","reported_seq":1493,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-22T03:13:18.185646+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:18.185646+0000","last_peered":"2024-09-22T03:13:18.185646+0000","last_clean":"2024-09-22T03:13:18.185646+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:18.185646+0000","last_undegraded":"2024-09-22T03:13:18.185646+0000","last_fullsized":"2024-09-22T03:13:18.185646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039218+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:20.039218+0000","last_peered":"2024-09-22T03:13:20.039218+0000","last_clean":"2024-09-22T03:13:20.039218+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:20.039218+0000","last_undegraded":"2024-09-22T03:13:20.039218+0000","last_fullsized":"2024-09-22T03:13:20.039218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"41'1463","reported_seq":2533,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039459+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:20.039459+0000","last_peered":"2024-09-22T03:13:20.039459+0000","last_clean":"2024-09-22T03:13:20.039459+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:20.039459+0000","last_undegraded":"2024-09-22T03:13:20.039459+0000","last_fullsized":"2024-09-22T03:13:20.039459+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1463,"log_dups_size":0,"ondisk_log_size":1463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1464,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.040632+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:20.040632+0000","last_peered":"2024-09-22T03:13:20.040632+0000","last_clean":"2024-09-22T03:13:20.040632+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:20.040632+0000","last_undegraded":"2024-09-22T03:13:20.040632+0000","last_fullsized":"2024-09-22T03:13:20.040632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":519045144,"num_objects":7921,"num_object_clones":0,"num_object_copies":23763,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":23231,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4579,"num_bytes_recovered":300089344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":92045312,"data_stored":1472528456,"data_compressed":69923525,"data_compressed_allocated":92033024,"data_compressed_original":1472528384,"omap_allocated":0,"internal_metadata":0},"log_size":23219,"ondisk_log_size":23219,"up":48,"acting":42,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673094,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82024,"kb_used_data":28820,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94289816,"statfs":{"total":96636764160,"available":96552771584,"internally_reserved":0,"allocated":29511680,"data_stored":466998021,"data_compressed":22169887,"data_compressed_allocated":29179904,"data_compressed_original":466878464,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673094,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79976,"kb_used_data":22616,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94291864,"statfs":{"total":96636764160,"available":96554868736,"internally_reserved":0,"allocated":23158784,"data_stored":365024525,"data_compressed":17329579,"data_compressed_allocated":22810624,"data_compressed_original":364888064,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149746,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72204,"kb_used_data":18360,"kb_used_omap":0,"kb_used_meta":53824,"kb_avail":94299636,"statfs":{"total":96636764160,"available":96562827264,"internally_reserved":0,"allocated":18800640,"data_stored":294973509,"data_compressed":14002851,"data_compressed_allocated":18432000,"data_compressed_original":294830080,"omap_allocated":0,"internal_metadata":55115776},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76176,"kb_used_data":22972,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94295664,"statfs":{"total":96636764160,"available":96558759936,"internally_reserved":0,"allocated":23523328,"data_stored":370984809,"data_compressed":17612772,"data_compressed_allocated":23183360,"data_compressed_original":370851840,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22585344,"data_stored":361299992,"data_compressed":17156456,"data_compressed_allocated":22581248,"data_compressed_original":361299968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18444288,"data_stored":295043096,"data_compressed":14010223,"data_compressed_allocated":18440192,"data_compressed_original":295043072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22355968,"data_stored":357695488,"data_compressed":16985295,"data_compressed_allocated":22355968,"data_compressed_original":357695488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28659712,"data_stored":458489880,"data_compressed":21771551,"data_compressed_allocated":28655616,"data_compressed_original":458489856,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:22.668 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:22.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:22.827+0000 7ff430e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:22.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:22.835+0000 7ff430e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:25.053 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:25.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:25.207+0000 7fd0d7797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:25.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:25.213+0000 7fd0d7797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:25.293 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:25.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:25.448+0000 7fd086f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:25.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:25.458+0000 7fd086f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:25.793 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:25.793 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:25.812 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-22T03:13:25.251644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":465371723,"num_objects":7097,"num_object_clones":0,"num_object_copies":21291,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":24198,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5172,"num_bytes_recovered":338952192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24148,"ondisk_log_size":24148,"up":75,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":314344,"kb_used_data":84256,"kb_used_omap":0,"kb_used_meta":230016,"kb_avail":377173016,"statfs":{"total":386547056640,"available":386225168384,"internally_reserved":0,"allocated":86278144,"data_stored":1357244206,"data_compressed":64430513,"data_compressed_allocated":84807680,"data_compressed_original":1356677120,"omap_allocated":0,"internal_metadata":235536384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-63635456,"num_objects":-971,"num_object_clones":0,"num_object_copies":-2913,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":-672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":971,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":672,"num_bytes_recovered":44040192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.242982"},"pg_stats":[{"pgid":"3.f","version":"42'1576","reported_seq":2299,"reported_epoch":42,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:22.218971+0000","last_change":"2024-09-22T03:13:22.201291+0000","last_active":"2024-09-22T03:13:22.218971+0000","last_peered":"2024-09-22T03:13:22.218971+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:22.218971+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1576,"log_dups_size":0,"ondisk_log_size":1576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":467,"num_object_clones":0,"num_object_copies":1401,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":21430272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"43'1534","reported_seq":2686,"reported_epoch":43,"state":"active+recovery_wait","last_fresh":"2024-09-22T03:13:23.224619+0000","last_change":"2024-09-22T03:13:18.131280+0000","last_active":"2024-09-22T03:13:23.224619+0000","last_peered":"2024-09-22T03:13:23.224619+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:23.224619+0000","last_undegraded":"2024-09-22T03:13:23.224619+0000","last_fullsized":"2024-09-22T03:13:23.224619+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1534,"log_dups_size":0,"ondisk_log_size":1534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28114944,"num_objects":429,"num_object_clones":0,"num_object_copies":1287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1535,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":["2","1","3"],"object_location_counts":[{"shards":"1,2,3","objects":429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"42'1486","reported_seq":2210,"reported_epoch":42,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:22.229675+0000","last_change":"2024-09-22T03:13:19.901424+0000","last_active":"2024-09-22T03:13:22.229675+0000","last_peered":"2024-09-22T03:13:22.229675+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:22.229675+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770328,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1486,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":21626880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"42'1543","reported_seq":1631,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:22.090093+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:22.090093+0000","last_peered":"2024-09-22T03:13:22.090093+0000","last_clean":"2024-09-22T03:13:22.090093+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:22.090093+0000","last_undegraded":"2024-09-22T03:13:22.090093+0000","last_fullsized":"2024-09-22T03:13:22.090093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1543,"log_dups_size":0,"ondisk_log_size":1543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":454,"num_object_clones":0,"num_object_copies":1362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1544,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"43'1511","reported_seq":1602,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.187991+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:23.187991+0000","last_peered":"2024-09-22T03:13:23.187991+0000","last_clean":"2024-09-22T03:13:23.187991+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:23.187991+0000","last_undegraded":"2024-09-22T03:13:23.187991+0000","last_fullsized":"2024-09-22T03:13:23.187991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1511,"log_dups_size":0,"ondisk_log_size":1511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1512,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"42'1497","reported_seq":2577,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:22.128320+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:22.128320+0000","last_peered":"2024-09-22T03:13:22.128320+0000","last_clean":"2024-09-22T03:13:22.128320+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:22.128320+0000","last_undegraded":"2024-09-22T03:13:22.128320+0000","last_fullsized":"2024-09-22T03:13:22.128320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1497,"log_dups_size":0,"ondisk_log_size":1497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":430,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1498,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"42'1498","reported_seq":2135,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:22.210419+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:22.210419+0000","last_peered":"2024-09-22T03:13:22.210419+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:22.210419+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1498,"log_dups_size":0,"ondisk_log_size":1498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27983872,"num_objects":427,"num_object_clones":0,"num_object_copies":1281,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":18546688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"40'1505","reported_seq":2590,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039583+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:20.039583+0000","last_peered":"2024-09-22T03:13:20.039583+0000","last_clean":"2024-09-22T03:13:20.039583+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:20.039583+0000","last_undegraded":"2024-09-22T03:13:20.039583+0000","last_fullsized":"2024-09-22T03:13:20.039583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1505,"log_dups_size":0,"ondisk_log_size":1505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":513,"num_object_clones":0,"num_object_copies":1539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1505,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.046684+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:23.046684+0000","last_peered":"2024-09-22T03:13:23.046684+0000","last_clean":"2024-09-22T03:13:23.046684+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:23.046684+0000","last_undegraded":"2024-09-22T03:13:23.046684+0000","last_fullsized":"2024-09-22T03:13:23.046684+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"40'1513","reported_seq":2578,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-22T03:13:20.036139+0000","last_change":"2024-09-22T03:13:20.113262+0000","last_active":"2024-09-22T03:13:20.113262+0000","last_peered":"2024-09-22T03:13:20.036139+0000","last_clean":"2024-09-22T03:11:38.298957+0000","last_became_active":"2024-09-22T03:12:32.542272+0000","last_became_peered":"2024-09-22T03:12:32.542272+0000","last_unstale":"2024-09-22T03:13:20.036139+0000","last_undegraded":"2024-09-22T03:13:20.036139+0000","last_fullsized":"2024-09-22T03:12:32.046866+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1513,"log_dups_size":0,"ondisk_log_size":1513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1513,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":107,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:21.018811+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:21.018811+0000","last_peered":"2024-09-22T03:13:21.018811+0000","last_clean":"2024-09-22T03:13:21.018811+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:21.018811+0000","last_undegraded":"2024-09-22T03:13:21.018811+0000","last_fullsized":"2024-09-22T03:13:21.018811+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"43'1486","reported_seq":1577,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.218911+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:23.218911+0000","last_peered":"2024-09-22T03:13:23.218911+0000","last_clean":"2024-09-22T03:13:23.218911+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:23.218911+0000","last_undegraded":"2024-09-22T03:13:23.218911+0000","last_fullsized":"2024-09-22T03:13:23.218911+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.034850+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:23.034850+0000","last_peered":"2024-09-22T03:13:23.034850+0000","last_clean":"2024-09-22T03:13:23.034850+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:23.034850+0000","last_undegraded":"2024-09-22T03:13:23.034850+0000","last_fullsized":"2024-09-22T03:13:23.034850+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"42'1536","reported_seq":2595,"reported_epoch":43,"state":"peering","last_fresh":"2024-09-22T03:13:22.864301+0000","last_change":"2024-09-22T03:13:23.049369+0000","last_active":"2024-09-22T03:13:23.049369+0000","last_peered":"2024-09-22T03:13:22.864301+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:22.864301+0000","last_undegraded":"2024-09-22T03:13:22.864301+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1536,"log_dups_size":0,"ondisk_log_size":1536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27525120,"num_objects":420,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1536,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.051271+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:23.051271+0000","last_peered":"2024-09-22T03:13:23.051271+0000","last_clean":"2024-09-22T03:13:23.051271+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:23.051271+0000","last_undegraded":"2024-09-22T03:13:23.051271+0000","last_fullsized":"2024-09-22T03:13:23.051271+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":231,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:21.022990+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:21.022990+0000","last_peered":"2024-09-22T03:13:21.022990+0000","last_clean":"2024-09-22T03:13:21.022990+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:21.022990+0000","last_undegraded":"2024-09-22T03:13:21.022990+0000","last_fullsized":"2024-09-22T03:13:21.022990+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"43'1481","reported_seq":2143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:23.236202+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:23.236202+0000","last_peered":"2024-09-22T03:13:23.236202+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:23.236202+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28704768,"num_objects":438,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1482,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":19398656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"43'1478","reported_seq":2403,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:23.247551+0000","last_change":"2024-09-22T03:13:16.793045+0000","last_active":"2024-09-22T03:13:23.247551+0000","last_peered":"2024-09-22T03:13:23.247551+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:23.247551+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1478,"log_dups_size":0,"ondisk_log_size":1478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":77,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":27918336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.041961+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:20.041961+0000","last_peered":"2024-09-22T03:13:20.041961+0000","last_clean":"2024-09-22T03:13:20.041961+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:20.041961+0000","last_undegraded":"2024-09-22T03:13:20.041961+0000","last_fullsized":"2024-09-22T03:13:20.041961+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"43'1464","reported_seq":2511,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.259195+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:23.259195+0000","last_peered":"2024-09-22T03:13:23.259195+0000","last_clean":"2024-09-22T03:13:23.259195+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:23.259195+0000","last_undegraded":"2024-09-22T03:13:23.259195+0000","last_fullsized":"2024-09-22T03:13:23.259195+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1464,"log_dups_size":0,"ondisk_log_size":1464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26542080,"num_objects":405,"num_object_clones":0,"num_object_copies":1215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.037769+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:23.037769+0000","last_peered":"2024-09-22T03:13:23.037769+0000","last_clean":"2024-09-22T03:13:23.037769+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:23.037769+0000","last_undegraded":"2024-09-22T03:13:23.037769+0000","last_fullsized":"2024-09-22T03:13:23.037769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"43'1477","reported_seq":1568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.167003+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:23.167003+0000","last_peered":"2024-09-22T03:13:23.167003+0000","last_clean":"2024-09-22T03:13:23.167003+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:23.167003+0000","last_undegraded":"2024-09-22T03:13:23.167003+0000","last_fullsized":"2024-09-22T03:13:23.167003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1477,"log_dups_size":0,"ondisk_log_size":1477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27525120,"num_objects":420,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039218+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:20.039218+0000","last_peered":"2024-09-22T03:13:20.039218+0000","last_clean":"2024-09-22T03:13:20.039218+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:20.039218+0000","last_undegraded":"2024-09-22T03:13:20.039218+0000","last_fullsized":"2024-09-22T03:13:20.039218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"41'1463","reported_seq":2533,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.039459+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:20.039459+0000","last_peered":"2024-09-22T03:13:20.039459+0000","last_clean":"2024-09-22T03:13:20.039459+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:20.039459+0000","last_undegraded":"2024-09-22T03:13:20.039459+0000","last_fullsized":"2024-09-22T03:13:20.039459+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1463,"log_dups_size":0,"ondisk_log_size":1463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1464,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-22T03:13:20.040632+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:20.040632+0000","last_peered":"2024-09-22T03:13:20.040632+0000","last_clean":"2024-09-22T03:13:20.040632+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:20.040632+0000","last_undegraded":"2024-09-22T03:13:20.040632+0000","last_fullsized":"2024-09-22T03:13:20.040632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":464781336,"num_objects":7093,"num_object_clones":0,"num_object_copies":21279,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":24059,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5172,"num_bytes_recovered":338952192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82743296,"data_stored":1323696200,"data_compressed":62856173,"data_compressed_allocated":82731008,"data_compressed_original":1323696128,"omap_allocated":0,"internal_metadata":0},"log_size":24048,"ondisk_log_size":24048,"up":48,"acting":43,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82804,"kb_used_data":25444,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94289036,"statfs":{"total":96636764160,"available":96551972864,"internally_reserved":0,"allocated":26054656,"data_stored":411302812,"data_compressed":19524687,"data_compressed_allocated":25698304,"data_compressed_original":411172864,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81508,"kb_used_data":19984,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94290332,"statfs":{"total":96636764160,"available":96553299968,"internally_reserved":0,"allocated":20463616,"data_stored":321519012,"data_compressed":15263211,"data_compressed_allocated":20090880,"data_compressed_original":321372160,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149746,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72204,"kb_used_data":18360,"kb_used_omap":0,"kb_used_meta":53824,"kb_avail":94299636,"statfs":{"total":96636764160,"available":96562827264,"internally_reserved":0,"allocated":18800640,"data_stored":294973509,"data_compressed":14002851,"data_compressed_allocated":18432000,"data_compressed_original":294830080,"omap_allocated":0,"internal_metadata":55115776},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77828,"kb_used_data":20468,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94294012,"statfs":{"total":96636764160,"available":96557068288,"internally_reserved":0,"allocated":20959232,"data_stored":329448873,"data_compressed":15639764,"data_compressed_allocated":20586496,"data_compressed_original":329302016,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20008960,"data_stored":320077848,"data_compressed":15199008,"data_compressed_allocated":20004864,"data_compressed_original":320077824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18444288,"data_stored":295043096,"data_compressed":14010223,"data_compressed_allocated":18440192,"data_compressed_original":295043072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19419136,"data_stored":310706176,"data_compressed":14753991,"data_compressed_allocated":19419136,"data_compressed_original":310706176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24870912,"data_stored":397869080,"data_compressed":18892951,"data_compressed_allocated":24866816,"data_compressed_original":397869056,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:25.813 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:26.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:25.999+0000 7f297e397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:26.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:26.010+0000 7f297e397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:26.507 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:26.507 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 802/21291 objects degraded (3.767%), 5 pgs degraded","count":5},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":151,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":45,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovering+undersized+degraded+remapped","count":3},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"peering","count":2},{"state_name":"active+recovery_wait","count":1}],"num_pgs":25,"num_pools":3,"num_objects":7097,"data_bytes":465371723,"bytes_used":321888256,"bytes_avail":386225168384,"bytes_total":386547056640,"inactive_pgs_ratio":0.079999998211860657,"degraded_objects":802,"degraded_total":21291,"degraded_ratio":0.037668498426565215,"recovering_objects_per_sec":128,"recovering_bytes_per_sec":8399836,"recovering_keys_per_sec":0,"num_objects_recovered":672,"num_bytes_recovered":44040192,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":185},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:26.527 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:26.527 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:26.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:26.665+0000 7f340062c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:26.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:26.673+0000 7f340062c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:26.965 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:26.965 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:26.979 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-22T03:13:26.031334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":451871307,"num_objects":6891,"num_object_clones":0,"num_object_copies":20673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":24404,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5172,"num_bytes_recovered":338952192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24352,"ondisk_log_size":24352,"up":75,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":322248,"kb_used_data":83844,"kb_used_omap":0,"kb_used_meta":238336,"kb_avail":377165112,"statfs":{"total":386547056640,"available":386217074688,"internally_reserved":0,"allocated":85856256,"data_stored":1350111086,"data_compressed":64091305,"data_compressed_allocated":84361216,"data_compressed_original":1349533696,"omap_allocated":0,"internal_metadata":244056064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-67764224,"num_objects":-1034,"num_object_clones":0,"num_object_copies":-3102,"num_objects_missing_on_primary":0,"num_objects_missing":-1,"num_objects_degraded":-593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1034,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":593,"num_bytes_recovered":38862848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.022520"},"pg_stats":[{"pgid":"3.f","version":"42'1576","reported_seq":2299,"reported_epoch":42,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:22.218971+0000","last_change":"2024-09-22T03:13:22.201291+0000","last_active":"2024-09-22T03:13:22.218971+0000","last_peered":"2024-09-22T03:13:22.218971+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:22.218971+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1576,"log_dups_size":0,"ondisk_log_size":1576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":467,"num_object_clones":0,"num_object_copies":1401,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":21430272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"43'1534","reported_seq":2686,"reported_epoch":43,"state":"active+recovery_wait","last_fresh":"2024-09-22T03:13:23.224619+0000","last_change":"2024-09-22T03:13:18.131280+0000","last_active":"2024-09-22T03:13:23.224619+0000","last_peered":"2024-09-22T03:13:23.224619+0000","last_clean":"2024-09-22T03:11:38.249528+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:23.224619+0000","last_undegraded":"2024-09-22T03:13:23.224619+0000","last_fullsized":"2024-09-22T03:13:23.224619+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1534,"log_dups_size":0,"ondisk_log_size":1534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28114944,"num_objects":429,"num_object_clones":0,"num_object_copies":1287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1535,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":["2","1","3"],"object_location_counts":[{"shards":"1,2,3","objects":429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"42'1486","reported_seq":2210,"reported_epoch":42,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:22.229675+0000","last_change":"2024-09-22T03:13:19.901424+0000","last_active":"2024-09-22T03:13:22.229675+0000","last_peered":"2024-09-22T03:13:22.229675+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:22.229675+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770328,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1486,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":21626880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"42'1543","reported_seq":1631,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:22.090093+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:22.090093+0000","last_peered":"2024-09-22T03:13:22.090093+0000","last_clean":"2024-09-22T03:13:22.090093+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:22.090093+0000","last_undegraded":"2024-09-22T03:13:22.090093+0000","last_fullsized":"2024-09-22T03:13:22.090093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1543,"log_dups_size":0,"ondisk_log_size":1543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":454,"num_object_clones":0,"num_object_copies":1362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1544,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"43'1511","reported_seq":1602,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.187991+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:23.187991+0000","last_peered":"2024-09-22T03:13:23.187991+0000","last_clean":"2024-09-22T03:13:23.187991+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:23.187991+0000","last_undegraded":"2024-09-22T03:13:23.187991+0000","last_fullsized":"2024-09-22T03:13:23.187991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1511,"log_dups_size":0,"ondisk_log_size":1511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1512,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"42'1497","reported_seq":2577,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:22.128320+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:22.128320+0000","last_peered":"2024-09-22T03:13:22.128320+0000","last_clean":"2024-09-22T03:13:22.128320+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:22.128320+0000","last_undegraded":"2024-09-22T03:13:22.128320+0000","last_fullsized":"2024-09-22T03:13:22.128320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1497,"log_dups_size":0,"ondisk_log_size":1497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":430,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1498,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"42'1498","reported_seq":2135,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:22.210419+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:22.210419+0000","last_peered":"2024-09-22T03:13:22.210419+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:22.210419+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1498,"log_dups_size":0,"ondisk_log_size":1498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27983872,"num_objects":427,"num_object_clones":0,"num_object_copies":1281,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":18546688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"44'1569","reported_seq":2664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:25.259158+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:25.259158+0000","last_peered":"2024-09-22T03:13:25.259158+0000","last_clean":"2024-09-22T03:13:25.259158+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:25.259158+0000","last_undegraded":"2024-09-22T03:13:25.259158+0000","last_fullsized":"2024-09-22T03:13:25.259158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1569,"log_dups_size":0,"ondisk_log_size":1569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":448,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1570,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.046684+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:23.046684+0000","last_peered":"2024-09-22T03:13:23.046684+0000","last_clean":"2024-09-22T03:13:23.046684+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:23.046684+0000","last_undegraded":"2024-09-22T03:13:23.046684+0000","last_fullsized":"2024-09-22T03:13:23.046684+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"44'1576","reported_seq":2664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:25.257269+0000","last_change":"2024-09-22T03:13:21.155496+0000","last_active":"2024-09-22T03:13:25.257269+0000","last_peered":"2024-09-22T03:13:25.257269+0000","last_clean":"2024-09-22T03:13:25.257269+0000","last_became_active":"2024-09-22T03:13:21.077994+0000","last_became_peered":"2024-09-22T03:13:21.077994+0000","last_unstale":"2024-09-22T03:13:25.257269+0000","last_undegraded":"2024-09-22T03:13:25.257269+0000","last_fullsized":"2024-09-22T03:13:25.257269+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1576,"log_dups_size":0,"ondisk_log_size":1576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27066368,"num_objects":413,"num_object_clones":0,"num_object_copies":1239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":107,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:21.018811+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:21.018811+0000","last_peered":"2024-09-22T03:13:21.018811+0000","last_clean":"2024-09-22T03:13:21.018811+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:21.018811+0000","last_undegraded":"2024-09-22T03:13:21.018811+0000","last_fullsized":"2024-09-22T03:13:21.018811+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"43'1486","reported_seq":1577,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.218911+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:23.218911+0000","last_peered":"2024-09-22T03:13:23.218911+0000","last_clean":"2024-09-22T03:13:23.218911+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:23.218911+0000","last_undegraded":"2024-09-22T03:13:23.218911+0000","last_fullsized":"2024-09-22T03:13:23.218911+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.034850+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:23.034850+0000","last_peered":"2024-09-22T03:13:23.034850+0000","last_clean":"2024-09-22T03:13:23.034850+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:23.034850+0000","last_undegraded":"2024-09-22T03:13:23.034850+0000","last_fullsized":"2024-09-22T03:13:23.034850+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"42'1536","reported_seq":2595,"reported_epoch":43,"state":"peering","last_fresh":"2024-09-22T03:13:22.864301+0000","last_change":"2024-09-22T03:13:23.049369+0000","last_active":"2024-09-22T03:13:23.049369+0000","last_peered":"2024-09-22T03:13:22.864301+0000","last_clean":"2024-09-22T03:11:38.299791+0000","last_became_active":"2024-09-22T03:12:32.537449+0000","last_became_peered":"2024-09-22T03:12:32.537449+0000","last_unstale":"2024-09-22T03:13:22.864301+0000","last_undegraded":"2024-09-22T03:13:22.864301+0000","last_fullsized":"2024-09-22T03:12:32.021371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1536,"log_dups_size":0,"ondisk_log_size":1536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27525120,"num_objects":420,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1536,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.051271+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:23.051271+0000","last_peered":"2024-09-22T03:13:23.051271+0000","last_clean":"2024-09-22T03:13:23.051271+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:23.051271+0000","last_undegraded":"2024-09-22T03:13:23.051271+0000","last_fullsized":"2024-09-22T03:13:23.051271+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":231,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-22T03:13:21.022990+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:21.022990+0000","last_peered":"2024-09-22T03:13:21.022990+0000","last_clean":"2024-09-22T03:13:21.022990+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:21.022990+0000","last_undegraded":"2024-09-22T03:13:21.022990+0000","last_fullsized":"2024-09-22T03:13:21.022990+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"43'1481","reported_seq":2143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:23.236202+0000","last_change":"2024-09-22T03:12:32.653299+0000","last_active":"2024-09-22T03:13:23.236202+0000","last_peered":"2024-09-22T03:13:23.236202+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:23.236202+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28704768,"num_objects":438,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1482,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":19398656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":262}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"43'1478","reported_seq":2403,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:23.247551+0000","last_change":"2024-09-22T03:13:16.793045+0000","last_active":"2024-09-22T03:13:23.247551+0000","last_peered":"2024-09-22T03:13:23.247551+0000","last_clean":"2024-09-22T03:11:38.333014+0000","last_became_active":"2024-09-22T03:12:32.563770+0000","last_became_peered":"2024-09-22T03:12:32.563770+0000","last_unstale":"2024-09-22T03:13:23.247551+0000","last_undegraded":"2024-09-22T03:12:32.045416+0000","last_fullsized":"2024-09-22T03:12:32.023507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1478,"log_dups_size":0,"ondisk_log_size":1478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":77,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":27918336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:24.044483+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:24.044483+0000","last_peered":"2024-09-22T03:13:24.044483+0000","last_clean":"2024-09-22T03:13:24.044483+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:24.044483+0000","last_undegraded":"2024-09-22T03:13:24.044483+0000","last_fullsized":"2024-09-22T03:13:24.044483+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"43'1464","reported_seq":2511,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.259195+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:23.259195+0000","last_peered":"2024-09-22T03:13:23.259195+0000","last_clean":"2024-09-22T03:13:23.259195+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:23.259195+0000","last_undegraded":"2024-09-22T03:13:23.259195+0000","last_fullsized":"2024-09-22T03:13:23.259195+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1464,"log_dups_size":0,"ondisk_log_size":1464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26542080,"num_objects":405,"num_object_clones":0,"num_object_copies":1215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.037769+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:23.037769+0000","last_peered":"2024-09-22T03:13:23.037769+0000","last_clean":"2024-09-22T03:13:23.037769+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:23.037769+0000","last_undegraded":"2024-09-22T03:13:23.037769+0000","last_fullsized":"2024-09-22T03:13:23.037769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"43'1477","reported_seq":1568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:13:23.167003+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:23.167003+0000","last_peered":"2024-09-22T03:13:23.167003+0000","last_clean":"2024-09-22T03:13:23.167003+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:23.167003+0000","last_undegraded":"2024-09-22T03:13:23.167003+0000","last_fullsized":"2024-09-22T03:13:23.167003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1477,"log_dups_size":0,"ondisk_log_size":1477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27525120,"num_objects":420,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:24.043732+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:24.043732+0000","last_peered":"2024-09-22T03:13:24.043732+0000","last_clean":"2024-09-22T03:13:24.043732+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:24.043732+0000","last_undegraded":"2024-09-22T03:13:24.043732+0000","last_fullsized":"2024-09-22T03:13:24.043732+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"44'1540","reported_seq":2619,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:25.125213+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:25.125213+0000","last_peered":"2024-09-22T03:13:25.125213+0000","last_clean":"2024-09-22T03:13:25.125213+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:25.125213+0000","last_undegraded":"2024-09-22T03:13:25.125213+0000","last_fullsized":"2024-09-22T03:13:25.125213+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1540,"log_dups_size":0,"ondisk_log_size":1540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26673152,"num_objects":407,"num_object_clones":0,"num_object_copies":1221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1541,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:24.044098+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:24.044098+0000","last_peered":"2024-09-22T03:13:24.044098+0000","last_clean":"2024-09-22T03:13:24.044098+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:24.044098+0000","last_undegraded":"2024-09-22T03:13:24.044098+0000","last_fullsized":"2024-09-22T03:13:24.044098+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":451280920,"num_objects":6887,"num_object_clones":0,"num_object_copies":20661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":24265,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5172,"num_bytes_recovered":338952192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82178048,"data_stored":1314652232,"data_compressed":62426717,"data_compressed_allocated":82165760,"data_compressed_original":1314652160,"omap_allocated":0,"internal_metadata":0},"log_size":24252,"ondisk_log_size":24252,"up":48,"acting":43,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82804,"kb_used_data":25444,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94289036,"statfs":{"total":96636764160,"available":96551972864,"internally_reserved":0,"allocated":26054656,"data_stored":411302812,"data_compressed":19524687,"data_compressed_allocated":25698304,"data_compressed_original":411172864,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81508,"kb_used_data":19984,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94290332,"statfs":{"total":96636764160,"available":96553299968,"internally_reserved":0,"allocated":20463616,"data_stored":321519012,"data_compressed":15263211,"data_compressed_allocated":20090880,"data_compressed_original":321372160,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":80108,"kb_used_data":17948,"kb_used_omap":0,"kb_used_meta":62144,"kb_avail":94291732,"statfs":{"total":96636764160,"available":96554733568,"internally_reserved":0,"allocated":18378752,"data_stored":287840389,"data_compressed":13663643,"data_compressed_allocated":17985536,"data_compressed_original":287686656,"omap_allocated":0,"internal_metadata":63635456},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77828,"kb_used_data":20468,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94294012,"statfs":{"total":96636764160,"available":96557068288,"internally_reserved":0,"allocated":20959232,"data_stored":329448873,"data_compressed":15639764,"data_compressed_allocated":20586496,"data_compressed_original":329302016,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20008960,"data_stored":320077848,"data_compressed":15199008,"data_compressed_allocated":20004864,"data_compressed_original":320077824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":285999128,"data_compressed":13580767,"data_compressed_allocated":17874944,"data_compressed_original":285999104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19419136,"data_stored":310706176,"data_compressed":14753991,"data_compressed_allocated":19419136,"data_compressed_original":310706176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24870912,"data_stored":397869080,"data_compressed":18892951,"data_compressed_allocated":24866816,"data_compressed_original":397869056,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:27.428 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:27.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:27.572+0000 7f7cfb46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:27.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:27.579+0000 7f7cfb46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:29.837 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:29.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:29.962+0000 7f8538957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:29.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:29.968+0000 7f8538957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:29.982 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:30.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:30.110+0000 7f8efbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:30.120+0000 7f8efbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:30.449 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:30.450 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:30.470 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-22T03:13:30.044223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393871947,"num_objects":6006,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":25289,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5724,"num_bytes_recovered":375128064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25236,"ondisk_log_size":25236,"up":75,"acting":71,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":270396,"kb_used_data":73592,"kb_used_omap":0,"kb_used_meta":196736,"kb_avail":377216964,"statfs":{"total":386547056640,"available":386270171136,"internally_reserved":0,"allocated":75358208,"data_stored":1180738105,"data_compressed":56046785,"data_compressed_allocated":73773056,"data_compressed_original":1180123136,"omap_allocated":0,"internal_metadata":201457664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-57999360,"num_objects":-885,"num_object_clones":0,"num_object_copies":-2655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-552,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":885,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":552,"num_bytes_recovered":36175872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012888"},"pg_stats":[{"pgid":"3.f","version":"46'1649","reported_seq":2708,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:27.212814+0000","last_change":"2024-09-22T03:13:22.201291+0000","last_active":"2024-09-22T03:13:27.212814+0000","last_peered":"2024-09-22T03:13:27.212814+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:27.212814+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1649,"log_dups_size":0,"ondisk_log_size":1649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":394,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1650,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":32178176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"47'1611","reported_seq":2775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.189796+0000","last_change":"2024-09-22T03:13:27.683865+0000","last_active":"2024-09-22T03:13:28.189796+0000","last_peered":"2024-09-22T03:13:28.189796+0000","last_clean":"2024-09-22T03:13:28.189796+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:28.189796+0000","last_undegraded":"2024-09-22T03:13:28.189796+0000","last_fullsized":"2024-09-22T03:13:28.189796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1611,"log_dups_size":0,"ondisk_log_size":1611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":352,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1612,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"46'1551","reported_seq":2580,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:27.222319+0000","last_change":"2024-09-22T03:13:19.901424+0000","last_active":"2024-09-22T03:13:27.222319+0000","last_peered":"2024-09-22T03:13:27.222319+0000","last_clean":"2024-09-22T03:11:38.211667+0000","last_became_active":"2024-09-22T03:12:32.547281+0000","last_became_peered":"2024-09-22T03:12:32.547281+0000","last_unstale":"2024-09-22T03:13:27.222319+0000","last_undegraded":"2024-09-22T03:12:32.166066+0000","last_fullsized":"2024-09-22T03:12:32.142159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444952,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1552,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":478,"num_bytes_recovered":31326208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"45'1613","reported_seq":1712,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.042926+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:27.042926+0000","last_peered":"2024-09-22T03:13:27.042926+0000","last_clean":"2024-09-22T03:13:27.042926+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:27.042926+0000","last_undegraded":"2024-09-22T03:13:27.042926+0000","last_fullsized":"2024-09-22T03:13:27.042926+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1613,"log_dups_size":0,"ondisk_log_size":1613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25231360,"num_objects":385,"num_object_clones":0,"num_object_copies":1155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1613,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"47'1593","reported_seq":1696,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.204704+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:28.204704+0000","last_peered":"2024-09-22T03:13:28.204704+0000","last_clean":"2024-09-22T03:13:28.204704+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:28.204704+0000","last_undegraded":"2024-09-22T03:13:28.204704+0000","last_fullsized":"2024-09-22T03:13:28.204704+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1593,"log_dups_size":0,"ondisk_log_size":1593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1594,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"46'1554","reported_seq":2646,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.171926+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:27.171926+0000","last_peered":"2024-09-22T03:13:27.171926+0000","last_clean":"2024-09-22T03:13:27.171926+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:27.171926+0000","last_undegraded":"2024-09-22T03:13:27.171926+0000","last_fullsized":"2024-09-22T03:13:27.171926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1554,"log_dups_size":0,"ondisk_log_size":1554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":373,"num_object_clones":0,"num_object_copies":1119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1555,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"45'1559","reported_seq":2326,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:27.231111+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:27.231111+0000","last_peered":"2024-09-22T03:13:27.231111+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:27.231111+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1559,"log_dups_size":0,"ondisk_log_size":1559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22544384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"44'1569","reported_seq":2664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:25.259158+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:25.259158+0000","last_peered":"2024-09-22T03:13:25.259158+0000","last_clean":"2024-09-22T03:13:25.259158+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:25.259158+0000","last_undegraded":"2024-09-22T03:13:25.259158+0000","last_fullsized":"2024-09-22T03:13:25.259158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1569,"log_dups_size":0,"ondisk_log_size":1569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":448,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1570,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.042356+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:28.042356+0000","last_peered":"2024-09-22T03:13:28.042356+0000","last_clean":"2024-09-22T03:13:28.042356+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:28.042356+0000","last_undegraded":"2024-09-22T03:13:28.042356+0000","last_fullsized":"2024-09-22T03:13:28.042356+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"44'1576","reported_seq":2664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:25.257269+0000","last_change":"2024-09-22T03:13:21.155496+0000","last_active":"2024-09-22T03:13:25.257269+0000","last_peered":"2024-09-22T03:13:25.257269+0000","last_clean":"2024-09-22T03:13:25.257269+0000","last_became_active":"2024-09-22T03:13:21.077994+0000","last_became_peered":"2024-09-22T03:13:21.077994+0000","last_unstale":"2024-09-22T03:13:25.257269+0000","last_undegraded":"2024-09-22T03:13:25.257269+0000","last_fullsized":"2024-09-22T03:13:25.257269+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1576,"log_dups_size":0,"ondisk_log_size":1576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27066368,"num_objects":413,"num_object_clones":0,"num_object_copies":1239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.039144+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:27.039144+0000","last_peered":"2024-09-22T03:13:27.039144+0000","last_clean":"2024-09-22T03:13:27.039144+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:27.039144+0000","last_undegraded":"2024-09-22T03:13:27.039144+0000","last_fullsized":"2024-09-22T03:13:27.039144+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"47'1555","reported_seq":1658,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.143055+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:28.143055+0000","last_peered":"2024-09-22T03:13:28.143055+0000","last_clean":"2024-09-22T03:13:28.143055+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:28.143055+0000","last_undegraded":"2024-09-22T03:13:28.143055+0000","last_fullsized":"2024-09-22T03:13:28.143055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.042891+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:28.042891+0000","last_peered":"2024-09-22T03:13:28.042891+0000","last_clean":"2024-09-22T03:13:28.042891+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:28.042891+0000","last_undegraded":"2024-09-22T03:13:28.042891+0000","last_fullsized":"2024-09-22T03:13:28.042891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"47'1596","reported_seq":2673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.186630+0000","last_change":"2024-09-22T03:13:24.076522+0000","last_active":"2024-09-22T03:13:28.186630+0000","last_peered":"2024-09-22T03:13:28.186630+0000","last_clean":"2024-09-22T03:13:28.186630+0000","last_became_active":"2024-09-22T03:13:24.069253+0000","last_became_peered":"2024-09-22T03:13:24.069253+0000","last_unstale":"2024-09-22T03:13:28.186630+0000","last_undegraded":"2024-09-22T03:13:28.186630+0000","last_fullsized":"2024-09-22T03:13:28.186630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1596,"log_dups_size":0,"ondisk_log_size":1596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23527424,"num_objects":359,"num_object_clones":0,"num_object_copies":1077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1597,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.045294+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:28.045294+0000","last_peered":"2024-09-22T03:13:28.045294+0000","last_clean":"2024-09-22T03:13:28.045294+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:28.045294+0000","last_undegraded":"2024-09-22T03:13:28.045294+0000","last_fullsized":"2024-09-22T03:13:28.045294+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":243,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.041768+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:27.041768+0000","last_peered":"2024-09-22T03:13:27.041768+0000","last_clean":"2024-09-22T03:13:27.041768+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:27.041768+0000","last_undegraded":"2024-09-22T03:13:27.041768+0000","last_fullsized":"2024-09-22T03:13:27.041768+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"47'1545","reported_seq":2419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:28.236421+0000","last_change":"2024-09-22T03:13:25.633701+0000","last_active":"2024-09-22T03:13:28.236421+0000","last_peered":"2024-09-22T03:13:28.236421+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:28.236421+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1545,"log_dups_size":0,"ondisk_log_size":1545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576000,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1545,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":26083328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":301}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"47'1555","reported_seq":2650,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.169746+0000","last_change":"2024-09-22T03:13:27.069795+0000","last_active":"2024-09-22T03:13:28.169746+0000","last_peered":"2024-09-22T03:13:28.169746+0000","last_clean":"2024-09-22T03:13:28.169746+0000","last_became_active":"2024-09-22T03:13:27.064201+0000","last_became_peered":"2024-09-22T03:13:27.064201+0000","last_unstale":"2024-09-22T03:13:28.169746+0000","last_undegraded":"2024-09-22T03:13:28.169746+0000","last_fullsized":"2024-09-22T03:13:28.169746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23330816,"num_objects":356,"num_object_clones":0,"num_object_copies":1068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":32964608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:24.044483+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:24.044483+0000","last_peered":"2024-09-22T03:13:24.044483+0000","last_clean":"2024-09-22T03:13:24.044483+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:24.044483+0000","last_undegraded":"2024-09-22T03:13:24.044483+0000","last_fullsized":"2024-09-22T03:13:24.044483+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'1524","reported_seq":2583,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.046326+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:28.046326+0000","last_peered":"2024-09-22T03:13:28.046326+0000","last_clean":"2024-09-22T03:13:28.046326+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:28.046326+0000","last_undegraded":"2024-09-22T03:13:28.046326+0000","last_fullsized":"2024-09-22T03:13:28.046326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1524,"log_dups_size":0,"ondisk_log_size":1524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1525,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.046153+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:28.046153+0000","last_peered":"2024-09-22T03:13:28.046153+0000","last_clean":"2024-09-22T03:13:28.046153+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:28.046153+0000","last_undegraded":"2024-09-22T03:13:28.046153+0000","last_fullsized":"2024-09-22T03:13:28.046153+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"47'1546","reported_seq":1649,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.253256+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:28.253256+0000","last_peered":"2024-09-22T03:13:28.253256+0000","last_clean":"2024-09-22T03:13:28.253256+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:28.253256+0000","last_undegraded":"2024-09-22T03:13:28.253256+0000","last_fullsized":"2024-09-22T03:13:28.253256+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1546,"log_dups_size":0,"ondisk_log_size":1546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23003136,"num_objects":351,"num_object_clones":0,"num_object_copies":1053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1547,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:24.043732+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:24.043732+0000","last_peered":"2024-09-22T03:13:24.043732+0000","last_clean":"2024-09-22T03:13:24.043732+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:24.043732+0000","last_undegraded":"2024-09-22T03:13:24.043732+0000","last_fullsized":"2024-09-22T03:13:24.043732+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"44'1540","reported_seq":2619,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:25.125213+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:25.125213+0000","last_peered":"2024-09-22T03:13:25.125213+0000","last_clean":"2024-09-22T03:13:25.125213+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:25.125213+0000","last_undegraded":"2024-09-22T03:13:25.125213+0000","last_fullsized":"2024-09-22T03:13:25.125213+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1540,"log_dups_size":0,"ondisk_log_size":1540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26673152,"num_objects":407,"num_object_clones":0,"num_object_copies":1221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1541,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T03:13:24.044098+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:24.044098+0000","last_peered":"2024-09-22T03:13:24.044098+0000","last_clean":"2024-09-22T03:13:24.044098+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:24.044098+0000","last_undegraded":"2024-09-22T03:13:24.044098+0000","last_fullsized":"2024-09-22T03:13:24.044098+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":393281560,"num_objects":6002,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":25150,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5724,"num_bytes_recovered":375128064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71913472,"data_stored":1150419016,"data_compressed":54628045,"data_compressed_allocated":71901184,"data_compressed_original":1150418944,"omap_allocated":0,"internal_metadata":0},"log_size":25136,"ondisk_log_size":25136,"up":48,"acting":44,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673104,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66404,"kb_used_data":21520,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94305436,"statfs":{"total":96636764160,"available":96568766464,"internally_reserved":0,"allocated":22036480,"data_stored":346501364,"data_compressed":16446919,"data_compressed_allocated":21647360,"data_compressed_original":346357760,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673104,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61788,"kb_used_data":16908,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94310052,"statfs":{"total":96636764160,"available":96573493248,"internally_reserved":0,"allocated":17313792,"data_stored":270611196,"data_compressed":12845187,"data_compressed_allocated":16908288,"data_compressed_original":270450688,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":80108,"kb_used_data":17948,"kb_used_omap":0,"kb_used_meta":62144,"kb_avail":94291732,"statfs":{"total":96636764160,"available":96554733568,"internally_reserved":0,"allocated":18378752,"data_stored":287840389,"data_compressed":13663643,"data_compressed_allocated":17985536,"data_compressed_original":287686656,"omap_allocated":0,"internal_metadata":63635456},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705808,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62096,"kb_used_data":17216,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94309744,"statfs":{"total":96636764160,"available":96573177856,"internally_reserved":0,"allocated":17629184,"data_stored":275785156,"data_compressed":13091036,"data_compressed_allocated":17231872,"data_compressed_original":275628032,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16961536,"data_stored":271319064,"data_compressed":12883680,"data_compressed_allocated":16957440,"data_compressed_original":271319040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":285999128,"data_compressed":13580767,"data_compressed_allocated":17874944,"data_compressed_original":285999104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16203776,"data_stored":259260416,"data_compressed":12311071,"data_compressed_allocated":16203776,"data_compressed_original":259260416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20869120,"data_stored":333840408,"data_compressed":15852527,"data_compressed_allocated":20865024,"data_compressed_original":333840384,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:30.472 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:30.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:30.665+0000 7f46c1f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:30.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:30.673+0000 7f46c1f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:31.165 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:31.165 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 250/18018 objects degraded (1.388%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":156,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":50,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovering+undersized+degraded+remapped","count":3},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":25,"num_pools":3,"num_objects":6006,"data_bytes":393871947,"bytes_used":276885504,"bytes_avail":386270171136,"bytes_total":386547056640,"degraded_objects":250,"degraded_total":18018,"degraded_ratio":0.013875013875013876,"recovering_objects_per_sec":137,"recovering_bytes_per_sec":9042218,"recovering_keys_per_sec":0,"num_objects_recovered":552,"num_bytes_recovered":36175872,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":272},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:31.184 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:31.185 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:31.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:31.309+0000 7f6b3406c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:31.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:31.315+0000 7f6b3406c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:31.493 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:31.493 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:31.505 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-22T03:13:31.252817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":377422411,"num_objects":5755,"num_object_clones":0,"num_object_copies":17265,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":241,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":25540,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5733,"num_bytes_recovered":375717888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25488,"ondisk_log_size":25488,"up":75,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257020,"kb_used_data":72696,"kb_used_omap":0,"kb_used_meta":184256,"kb_avail":377230340,"statfs":{"total":386547056640,"available":386283868160,"internally_reserved":0,"allocated":74440704,"data_stored":1165419552,"data_compressed":55318577,"data_compressed_allocated":72814592,"data_compressed_original":1164787712,"omap_allocated":0,"internal_metadata":188678144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74448896,"num_objects":-1136,"num_object_clones":0,"num_object_copies":-3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-561,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1136,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":561,"num_bytes_recovered":36765696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000794"},"pg_stats":[{"pgid":"3.f","version":"46'1649","reported_seq":2708,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:27.212814+0000","last_change":"2024-09-22T03:13:22.201291+0000","last_active":"2024-09-22T03:13:27.212814+0000","last_peered":"2024-09-22T03:13:27.212814+0000","last_clean":"2024-09-22T03:11:38.338625+0000","last_became_active":"2024-09-22T03:12:32.583500+0000","last_became_peered":"2024-09-22T03:12:32.583500+0000","last_unstale":"2024-09-22T03:13:27.212814+0000","last_undegraded":"2024-09-22T03:12:32.139030+0000","last_fullsized":"2024-09-22T03:12:32.108318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1649,"log_dups_size":0,"ondisk_log_size":1649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":394,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1650,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":32178176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"47'1611","reported_seq":2775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.189796+0000","last_change":"2024-09-22T03:13:27.683865+0000","last_active":"2024-09-22T03:13:28.189796+0000","last_peered":"2024-09-22T03:13:28.189796+0000","last_clean":"2024-09-22T03:13:28.189796+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:28.189796+0000","last_undegraded":"2024-09-22T03:13:28.189796+0000","last_fullsized":"2024-09-22T03:13:28.189796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1611,"log_dups_size":0,"ondisk_log_size":1611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":352,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1612,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'1598","reported_seq":2660,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.280901+0000","last_change":"2024-09-22T03:13:29.087178+0000","last_active":"2024-09-22T03:13:30.280901+0000","last_peered":"2024-09-22T03:13:30.280901+0000","last_clean":"2024-09-22T03:13:30.280901+0000","last_became_active":"2024-09-22T03:13:29.071173+0000","last_became_peered":"2024-09-22T03:13:29.071173+0000","last_unstale":"2024-09-22T03:13:30.280901+0000","last_undegraded":"2024-09-22T03:13:30.280901+0000","last_fullsized":"2024-09-22T03:13:30.280901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1598,"log_dups_size":0,"ondisk_log_size":1598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364760,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1599,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":31916032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"45'1613","reported_seq":1712,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.042926+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:27.042926+0000","last_peered":"2024-09-22T03:13:27.042926+0000","last_clean":"2024-09-22T03:13:27.042926+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:27.042926+0000","last_undegraded":"2024-09-22T03:13:27.042926+0000","last_fullsized":"2024-09-22T03:13:27.042926+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1613,"log_dups_size":0,"ondisk_log_size":1613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25231360,"num_objects":385,"num_object_clones":0,"num_object_copies":1155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1613,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"47'1593","reported_seq":1696,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.204704+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:28.204704+0000","last_peered":"2024-09-22T03:13:28.204704+0000","last_clean":"2024-09-22T03:13:28.204704+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:28.204704+0000","last_undegraded":"2024-09-22T03:13:28.204704+0000","last_fullsized":"2024-09-22T03:13:28.204704+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1593,"log_dups_size":0,"ondisk_log_size":1593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1594,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"46'1554","reported_seq":2646,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.171926+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:27.171926+0000","last_peered":"2024-09-22T03:13:27.171926+0000","last_clean":"2024-09-22T03:13:27.171926+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:27.171926+0000","last_undegraded":"2024-09-22T03:13:27.171926+0000","last_fullsized":"2024-09-22T03:13:27.171926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1554,"log_dups_size":0,"ondisk_log_size":1554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":373,"num_object_clones":0,"num_object_copies":1119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1555,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"45'1559","reported_seq":2326,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:27.231111+0000","last_change":"2024-09-22T03:12:32.662346+0000","last_active":"2024-09-22T03:13:27.231111+0000","last_peered":"2024-09-22T03:13:27.231111+0000","last_clean":"2024-09-22T03:11:38.299120+0000","last_became_active":"2024-09-22T03:12:32.536956+0000","last_became_peered":"2024-09-22T03:12:32.536956+0000","last_unstale":"2024-09-22T03:13:27.231111+0000","last_undegraded":"2024-09-22T03:12:32.044280+0000","last_fullsized":"2024-09-22T03:12:32.024236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1559,"log_dups_size":0,"ondisk_log_size":1559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22544384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":243}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1641","reported_seq":2751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.259635+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:30.259635+0000","last_peered":"2024-09-22T03:13:30.259635+0000","last_clean":"2024-09-22T03:13:30.259635+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:30.259635+0000","last_undegraded":"2024-09-22T03:13:30.259635+0000","last_fullsized":"2024-09-22T03:13:30.259635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1641,"log_dups_size":0,"ondisk_log_size":1641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641536,"num_objects":376,"num_object_clones":0,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.042356+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:28.042356+0000","last_peered":"2024-09-22T03:13:28.042356+0000","last_clean":"2024-09-22T03:13:28.042356+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:28.042356+0000","last_undegraded":"2024-09-22T03:13:28.042356+0000","last_fullsized":"2024-09-22T03:13:28.042356+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"49'1645","reported_seq":2748,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.288834+0000","last_change":"2024-09-22T03:13:21.155496+0000","last_active":"2024-09-22T03:13:30.288834+0000","last_peered":"2024-09-22T03:13:30.288834+0000","last_clean":"2024-09-22T03:13:30.288834+0000","last_became_active":"2024-09-22T03:13:21.077994+0000","last_became_peered":"2024-09-22T03:13:21.077994+0000","last_unstale":"2024-09-22T03:13:30.288834+0000","last_undegraded":"2024-09-22T03:13:30.288834+0000","last_fullsized":"2024-09-22T03:13:30.288834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1645,"log_dups_size":0,"ondisk_log_size":1645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1646,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.039144+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:27.039144+0000","last_peered":"2024-09-22T03:13:27.039144+0000","last_clean":"2024-09-22T03:13:27.039144+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:27.039144+0000","last_undegraded":"2024-09-22T03:13:27.039144+0000","last_fullsized":"2024-09-22T03:13:27.039144+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"47'1555","reported_seq":1658,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.143055+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:28.143055+0000","last_peered":"2024-09-22T03:13:28.143055+0000","last_clean":"2024-09-22T03:13:28.143055+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:28.143055+0000","last_undegraded":"2024-09-22T03:13:28.143055+0000","last_fullsized":"2024-09-22T03:13:28.143055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.042891+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:28.042891+0000","last_peered":"2024-09-22T03:13:28.042891+0000","last_clean":"2024-09-22T03:13:28.042891+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:28.042891+0000","last_undegraded":"2024-09-22T03:13:28.042891+0000","last_fullsized":"2024-09-22T03:13:28.042891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"47'1596","reported_seq":2673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.186630+0000","last_change":"2024-09-22T03:13:24.076522+0000","last_active":"2024-09-22T03:13:28.186630+0000","last_peered":"2024-09-22T03:13:28.186630+0000","last_clean":"2024-09-22T03:13:28.186630+0000","last_became_active":"2024-09-22T03:13:24.069253+0000","last_became_peered":"2024-09-22T03:13:24.069253+0000","last_unstale":"2024-09-22T03:13:28.186630+0000","last_undegraded":"2024-09-22T03:13:28.186630+0000","last_fullsized":"2024-09-22T03:13:28.186630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1596,"log_dups_size":0,"ondisk_log_size":1596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23527424,"num_objects":359,"num_object_clones":0,"num_object_copies":1077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1597,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.045294+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:28.045294+0000","last_peered":"2024-09-22T03:13:28.045294+0000","last_clean":"2024-09-22T03:13:28.045294+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:28.045294+0000","last_undegraded":"2024-09-22T03:13:28.045294+0000","last_fullsized":"2024-09-22T03:13:28.045294+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":243,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:13:27.041768+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:27.041768+0000","last_peered":"2024-09-22T03:13:27.041768+0000","last_clean":"2024-09-22T03:13:27.041768+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:27.041768+0000","last_undegraded":"2024-09-22T03:13:27.041768+0000","last_fullsized":"2024-09-22T03:13:27.041768+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"47'1545","reported_seq":2419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:28.236421+0000","last_change":"2024-09-22T03:13:25.633701+0000","last_active":"2024-09-22T03:13:28.236421+0000","last_peered":"2024-09-22T03:13:28.236421+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:28.236421+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1545,"log_dups_size":0,"ondisk_log_size":1545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576000,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1545,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":26083328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":301}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"47'1555","reported_seq":2650,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.169746+0000","last_change":"2024-09-22T03:13:27.069795+0000","last_active":"2024-09-22T03:13:28.169746+0000","last_peered":"2024-09-22T03:13:28.169746+0000","last_clean":"2024-09-22T03:13:28.169746+0000","last_became_active":"2024-09-22T03:13:27.064201+0000","last_became_peered":"2024-09-22T03:13:27.064201+0000","last_unstale":"2024-09-22T03:13:28.169746+0000","last_undegraded":"2024-09-22T03:13:28.169746+0000","last_fullsized":"2024-09-22T03:13:28.169746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23330816,"num_objects":356,"num_object_clones":0,"num_object_copies":1068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":32964608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.061459+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:30.061459+0000","last_peered":"2024-09-22T03:13:30.061459+0000","last_clean":"2024-09-22T03:13:30.061459+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:30.061459+0000","last_undegraded":"2024-09-22T03:13:30.061459+0000","last_fullsized":"2024-09-22T03:13:30.061459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'1524","reported_seq":2583,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.046326+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:28.046326+0000","last_peered":"2024-09-22T03:13:28.046326+0000","last_clean":"2024-09-22T03:13:28.046326+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:28.046326+0000","last_undegraded":"2024-09-22T03:13:28.046326+0000","last_fullsized":"2024-09-22T03:13:28.046326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1524,"log_dups_size":0,"ondisk_log_size":1524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1525,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.046153+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:28.046153+0000","last_peered":"2024-09-22T03:13:28.046153+0000","last_clean":"2024-09-22T03:13:28.046153+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:28.046153+0000","last_undegraded":"2024-09-22T03:13:28.046153+0000","last_fullsized":"2024-09-22T03:13:28.046153+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"47'1546","reported_seq":1649,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.253256+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:28.253256+0000","last_peered":"2024-09-22T03:13:28.253256+0000","last_clean":"2024-09-22T03:13:28.253256+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:28.253256+0000","last_undegraded":"2024-09-22T03:13:28.253256+0000","last_fullsized":"2024-09-22T03:13:28.253256+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1546,"log_dups_size":0,"ondisk_log_size":1546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23003136,"num_objects":351,"num_object_clones":0,"num_object_copies":1053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1547,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.060862+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:30.060862+0000","last_peered":"2024-09-22T03:13:30.060862+0000","last_clean":"2024-09-22T03:13:30.060862+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:30.060862+0000","last_undegraded":"2024-09-22T03:13:30.060862+0000","last_fullsized":"2024-09-22T03:13:30.060862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"48'1604","reported_seq":2697,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.056381+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:30.056381+0000","last_peered":"2024-09-22T03:13:30.056381+0000","last_clean":"2024-09-22T03:13:30.056381+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:30.056381+0000","last_undegraded":"2024-09-22T03:13:30.056381+0000","last_fullsized":"2024-09-22T03:13:30.056381+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1604,"log_dups_size":0,"ondisk_log_size":1604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.061228+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:30.061228+0000","last_peered":"2024-09-22T03:13:30.061228+0000","last_clean":"2024-09-22T03:13:30.061228+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:30.061228+0000","last_undegraded":"2024-09-22T03:13:30.061228+0000","last_fullsized":"2024-09-22T03:13:30.061228+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":376832024,"num_objects":5751,"num_object_clones":0,"num_object_copies":17253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":241,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":25401,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5733,"num_bytes_recovered":375717888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70832128,"data_stored":1133117512,"data_compressed":53806477,"data_compressed_allocated":70819840,"data_compressed_original":1133117440,"omap_allocated":0,"internal_metadata":0},"log_size":25388,"ondisk_log_size":25388,"up":48,"acting":45,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673104,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66404,"kb_used_data":21520,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94305436,"statfs":{"total":96636764160,"available":96568766464,"internally_reserved":0,"allocated":22036480,"data_stored":346501364,"data_compressed":16446919,"data_compressed_allocated":21647360,"data_compressed_original":346357760,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673104,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61788,"kb_used_data":16908,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94310052,"statfs":{"total":96636764160,"available":96573493248,"internally_reserved":0,"allocated":17313792,"data_stored":270611196,"data_compressed":12845187,"data_compressed_allocated":16908288,"data_compressed_original":270450688,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149756,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66732,"kb_used_data":17052,"kb_used_omap":0,"kb_used_meta":49664,"kb_avail":94305108,"statfs":{"total":96636764160,"available":96568430592,"internally_reserved":0,"allocated":17461248,"data_stored":272521836,"data_compressed":12935435,"data_compressed_allocated":17027072,"data_compressed_original":272351232,"omap_allocated":0,"internal_metadata":50855936},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705808,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62096,"kb_used_data":17216,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94309744,"statfs":{"total":96636764160,"available":96573177856,"internally_reserved":0,"allocated":17629184,"data_stored":275785156,"data_compressed":13091036,"data_compressed_allocated":17231872,"data_compressed_original":275628032,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16961536,"data_stored":271319064,"data_compressed":12883680,"data_compressed_allocated":16957440,"data_compressed_original":271319040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16797696,"data_stored":268697624,"data_compressed":12759199,"data_compressed_allocated":16793600,"data_compressed_original":268697600,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16203776,"data_stored":259260416,"data_compressed":12311071,"data_compressed_allocated":16203776,"data_compressed_original":259260416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20869120,"data_stored":333840408,"data_compressed":15852527,"data_compressed_allocated":20865024,"data_compressed_original":333840384,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:32.092 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:32.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:32.240+0000 7ffb8dd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:32.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:32.246+0000 7ffb8dd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:34.502 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:34.507 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:34.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:34.651+0000 7fefafd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:34.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:34.659+0000 7fefafd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:34.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:34.700+0000 7f0de2197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:34.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:34.712+0000 7f0de2197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:34.972 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:34.972 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:34.986 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-22T03:13:33.253222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362480203,"num_objects":5527,"num_object_clones":0,"num_object_copies":16581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":25768,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5900,"num_bytes_recovered":386662400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25716,"ondisk_log_size":25716,"up":75,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":254436,"kb_used_data":70112,"kb_used_omap":0,"kb_used_meta":184256,"kb_avail":377232924,"statfs":{"total":386547056640,"available":386286514176,"internally_reserved":0,"allocated":71794688,"data_stored":1122444720,"data_compressed":53277105,"data_compressed_allocated":70127616,"data_compressed_original":1121796096,"omap_allocated":0,"internal_metadata":188678144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31391744,"num_objects":-479,"num_object_clones":0,"num_object_copies":-1437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":479,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":11534336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.215552"},"pg_stats":[{"pgid":"3.f","version":"51'1708","reported_seq":2873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.213646+0000","last_change":"2024-09-22T03:13:30.111709+0000","last_active":"2024-09-22T03:13:32.213646+0000","last_peered":"2024-09-22T03:13:32.213646+0000","last_clean":"2024-09-22T03:13:32.213646+0000","last_became_active":"2024-09-22T03:13:30.105355+0000","last_became_peered":"2024-09-22T03:13:30.105355+0000","last_unstale":"2024-09-22T03:13:32.213646+0000","last_undegraded":"2024-09-22T03:13:32.213646+0000","last_fullsized":"2024-09-22T03:13:32.213646+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21954560,"num_objects":335,"num_object_clones":0,"num_object_copies":1005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1709,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":35061760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"47'1611","reported_seq":2775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.189796+0000","last_change":"2024-09-22T03:13:27.683865+0000","last_active":"2024-09-22T03:13:28.189796+0000","last_peered":"2024-09-22T03:13:28.189796+0000","last_clean":"2024-09-22T03:13:28.189796+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:28.189796+0000","last_undegraded":"2024-09-22T03:13:28.189796+0000","last_fullsized":"2024-09-22T03:13:28.189796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1611,"log_dups_size":0,"ondisk_log_size":1611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":352,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1612,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'1598","reported_seq":2660,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.280901+0000","last_change":"2024-09-22T03:13:29.087178+0000","last_active":"2024-09-22T03:13:30.280901+0000","last_peered":"2024-09-22T03:13:30.280901+0000","last_clean":"2024-09-22T03:13:30.280901+0000","last_became_active":"2024-09-22T03:13:29.071173+0000","last_became_peered":"2024-09-22T03:13:29.071173+0000","last_unstale":"2024-09-22T03:13:30.280901+0000","last_undegraded":"2024-09-22T03:13:30.280901+0000","last_fullsized":"2024-09-22T03:13:30.280901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1598,"log_dups_size":0,"ondisk_log_size":1598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364760,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1599,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":31916032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"51'1674","reported_seq":1789,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.181111+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:32.181111+0000","last_peered":"2024-09-22T03:13:32.181111+0000","last_clean":"2024-09-22T03:13:32.181111+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:32.181111+0000","last_undegraded":"2024-09-22T03:13:32.181111+0000","last_fullsized":"2024-09-22T03:13:32.181111+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1674,"log_dups_size":0,"ondisk_log_size":1674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1675,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"47'1593","reported_seq":1696,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.204704+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:28.204704+0000","last_peered":"2024-09-22T03:13:28.204704+0000","last_clean":"2024-09-22T03:13:28.204704+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:28.204704+0000","last_undegraded":"2024-09-22T03:13:28.204704+0000","last_fullsized":"2024-09-22T03:13:28.204704+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1593,"log_dups_size":0,"ondisk_log_size":1593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1594,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"50'1604","reported_seq":2710,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060484+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:32.060484+0000","last_peered":"2024-09-22T03:13:32.060484+0000","last_clean":"2024-09-22T03:13:32.060484+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:32.060484+0000","last_undegraded":"2024-09-22T03:13:32.060484+0000","last_fullsized":"2024-09-22T03:13:32.060484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1604,"log_dups_size":0,"ondisk_log_size":1604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"51'1617","reported_seq":2647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.123645+0000","last_change":"2024-09-22T03:13:32.077980+0000","last_active":"2024-09-22T03:13:32.123645+0000","last_peered":"2024-09-22T03:13:32.123645+0000","last_clean":"2024-09-22T03:13:32.123645+0000","last_became_active":"2024-09-22T03:13:32.073222+0000","last_became_peered":"2024-09-22T03:13:32.073222+0000","last_unstale":"2024-09-22T03:13:32.123645+0000","last_undegraded":"2024-09-22T03:13:32.123645+0000","last_fullsized":"2024-09-22T03:13:32.123645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1617,"log_dups_size":0,"ondisk_log_size":1617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1618,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":467,"num_bytes_recovered":30605312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1641","reported_seq":2751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.259635+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:30.259635+0000","last_peered":"2024-09-22T03:13:30.259635+0000","last_clean":"2024-09-22T03:13:30.259635+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:30.259635+0000","last_undegraded":"2024-09-22T03:13:30.259635+0000","last_fullsized":"2024-09-22T03:13:30.259635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1641,"log_dups_size":0,"ondisk_log_size":1641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641536,"num_objects":376,"num_object_clones":0,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.042356+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:28.042356+0000","last_peered":"2024-09-22T03:13:28.042356+0000","last_clean":"2024-09-22T03:13:28.042356+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:28.042356+0000","last_undegraded":"2024-09-22T03:13:28.042356+0000","last_fullsized":"2024-09-22T03:13:28.042356+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"49'1645","reported_seq":2748,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.288834+0000","last_change":"2024-09-22T03:13:21.155496+0000","last_active":"2024-09-22T03:13:30.288834+0000","last_peered":"2024-09-22T03:13:30.288834+0000","last_clean":"2024-09-22T03:13:30.288834+0000","last_became_active":"2024-09-22T03:13:21.077994+0000","last_became_peered":"2024-09-22T03:13:21.077994+0000","last_unstale":"2024-09-22T03:13:30.288834+0000","last_undegraded":"2024-09-22T03:13:30.288834+0000","last_fullsized":"2024-09-22T03:13:30.288834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1645,"log_dups_size":0,"ondisk_log_size":1645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1646,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.057686+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:32.057686+0000","last_peered":"2024-09-22T03:13:32.057686+0000","last_clean":"2024-09-22T03:13:32.057686+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:32.057686+0000","last_undegraded":"2024-09-22T03:13:32.057686+0000","last_fullsized":"2024-09-22T03:13:32.057686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"47'1555","reported_seq":1658,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.143055+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:28.143055+0000","last_peered":"2024-09-22T03:13:28.143055+0000","last_clean":"2024-09-22T03:13:28.143055+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:28.143055+0000","last_undegraded":"2024-09-22T03:13:28.143055+0000","last_fullsized":"2024-09-22T03:13:28.143055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.042891+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:28.042891+0000","last_peered":"2024-09-22T03:13:28.042891+0000","last_clean":"2024-09-22T03:13:28.042891+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:28.042891+0000","last_undegraded":"2024-09-22T03:13:28.042891+0000","last_fullsized":"2024-09-22T03:13:28.042891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"47'1596","reported_seq":2673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.186630+0000","last_change":"2024-09-22T03:13:24.076522+0000","last_active":"2024-09-22T03:13:28.186630+0000","last_peered":"2024-09-22T03:13:28.186630+0000","last_clean":"2024-09-22T03:13:28.186630+0000","last_became_active":"2024-09-22T03:13:24.069253+0000","last_became_peered":"2024-09-22T03:13:24.069253+0000","last_unstale":"2024-09-22T03:13:28.186630+0000","last_undegraded":"2024-09-22T03:13:28.186630+0000","last_fullsized":"2024-09-22T03:13:28.186630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1596,"log_dups_size":0,"ondisk_log_size":1596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23527424,"num_objects":359,"num_object_clones":0,"num_object_copies":1077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1597,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.045294+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:28.045294+0000","last_peered":"2024-09-22T03:13:28.045294+0000","last_clean":"2024-09-22T03:13:28.045294+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:28.045294+0000","last_undegraded":"2024-09-22T03:13:28.045294+0000","last_fullsized":"2024-09-22T03:13:28.045294+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":258,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060174+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:32.060174+0000","last_peered":"2024-09-22T03:13:32.060174+0000","last_clean":"2024-09-22T03:13:32.060174+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:32.060174+0000","last_undegraded":"2024-09-22T03:13:32.060174+0000","last_fullsized":"2024-09-22T03:13:32.060174+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"47'1545","reported_seq":2419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:13:28.236421+0000","last_change":"2024-09-22T03:13:25.633701+0000","last_active":"2024-09-22T03:13:28.236421+0000","last_peered":"2024-09-22T03:13:28.236421+0000","last_clean":"2024-09-22T03:11:38.280052+0000","last_became_active":"2024-09-22T03:12:32.645652+0000","last_became_peered":"2024-09-22T03:12:32.645652+0000","last_unstale":"2024-09-22T03:13:28.236421+0000","last_undegraded":"2024-09-22T03:12:32.369269+0000","last_fullsized":"2024-09-22T03:12:32.348376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1545,"log_dups_size":0,"ondisk_log_size":1545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576000,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1545,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":26083328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":301}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"47'1555","reported_seq":2650,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.169746+0000","last_change":"2024-09-22T03:13:27.069795+0000","last_active":"2024-09-22T03:13:28.169746+0000","last_peered":"2024-09-22T03:13:28.169746+0000","last_clean":"2024-09-22T03:13:28.169746+0000","last_became_active":"2024-09-22T03:13:27.064201+0000","last_became_peered":"2024-09-22T03:13:27.064201+0000","last_unstale":"2024-09-22T03:13:28.169746+0000","last_undegraded":"2024-09-22T03:13:28.169746+0000","last_fullsized":"2024-09-22T03:13:28.169746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23330816,"num_objects":356,"num_object_clones":0,"num_object_copies":1068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":32964608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.061459+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:30.061459+0000","last_peered":"2024-09-22T03:13:30.061459+0000","last_clean":"2024-09-22T03:13:30.061459+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:30.061459+0000","last_undegraded":"2024-09-22T03:13:30.061459+0000","last_fullsized":"2024-09-22T03:13:30.061459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'1524","reported_seq":2583,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.046326+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:28.046326+0000","last_peered":"2024-09-22T03:13:28.046326+0000","last_clean":"2024-09-22T03:13:28.046326+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:28.046326+0000","last_undegraded":"2024-09-22T03:13:28.046326+0000","last_fullsized":"2024-09-22T03:13:28.046326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1524,"log_dups_size":0,"ondisk_log_size":1524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1525,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.046153+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:28.046153+0000","last_peered":"2024-09-22T03:13:28.046153+0000","last_clean":"2024-09-22T03:13:28.046153+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:28.046153+0000","last_undegraded":"2024-09-22T03:13:28.046153+0000","last_fullsized":"2024-09-22T03:13:28.046153+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"47'1546","reported_seq":1649,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-22T03:13:28.253256+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:28.253256+0000","last_peered":"2024-09-22T03:13:28.253256+0000","last_clean":"2024-09-22T03:13:28.253256+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:28.253256+0000","last_undegraded":"2024-09-22T03:13:28.253256+0000","last_fullsized":"2024-09-22T03:13:28.253256+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1546,"log_dups_size":0,"ondisk_log_size":1546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23003136,"num_objects":351,"num_object_clones":0,"num_object_copies":1053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1547,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.060862+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:30.060862+0000","last_peered":"2024-09-22T03:13:30.060862+0000","last_clean":"2024-09-22T03:13:30.060862+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:30.060862+0000","last_undegraded":"2024-09-22T03:13:30.060862+0000","last_fullsized":"2024-09-22T03:13:30.060862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"48'1604","reported_seq":2697,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.056381+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:30.056381+0000","last_peered":"2024-09-22T03:13:30.056381+0000","last_clean":"2024-09-22T03:13:30.056381+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:30.056381+0000","last_undegraded":"2024-09-22T03:13:30.056381+0000","last_fullsized":"2024-09-22T03:13:30.056381+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1604,"log_dups_size":0,"ondisk_log_size":1604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.061228+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:30.061228+0000","last_peered":"2024-09-22T03:13:30.061228+0000","last_clean":"2024-09-22T03:13:30.061228+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:30.061228+0000","last_undegraded":"2024-09-22T03:13:30.061228+0000","last_fullsized":"2024-09-22T03:13:30.061228+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":361889816,"num_objects":5523,"num_object_clones":0,"num_object_copies":16569,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":25629,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5900,"num_bytes_recovered":386662400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68210688,"data_stored":1091174472,"data_compressed":51814797,"data_compressed_allocated":68198400,"data_compressed_original":1091174400,"omap_allocated":0,"internal_metadata":0},"log_size":25616,"ondisk_log_size":25616,"up":48,"acting":47,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673104,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66404,"kb_used_data":21520,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94305436,"statfs":{"total":96636764160,"available":96568766464,"internally_reserved":0,"allocated":22036480,"data_stored":346501364,"data_compressed":16446919,"data_compressed_allocated":21647360,"data_compressed_original":346357760,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673104,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61788,"kb_used_data":16908,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94310052,"statfs":{"total":96636764160,"available":96573493248,"internally_reserved":0,"allocated":17313792,"data_stored":270611196,"data_compressed":12845187,"data_compressed_allocated":16908288,"data_compressed_original":270450688,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149756,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66732,"kb_used_data":17052,"kb_used_omap":0,"kb_used_meta":49664,"kb_avail":94305108,"statfs":{"total":96636764160,"available":96568430592,"internally_reserved":0,"allocated":17461248,"data_stored":272521836,"data_compressed":12935435,"data_compressed_allocated":17027072,"data_compressed_original":272351232,"omap_allocated":0,"internal_metadata":50855936},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59512,"kb_used_data":14632,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94312328,"statfs":{"total":96636764160,"available":96575823872,"internally_reserved":0,"allocated":14983168,"data_stored":232810324,"data_compressed":11049564,"data_compressed_allocated":14544896,"data_compressed_original":232636416,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14340096,"data_stored":229376024,"data_compressed":10892000,"data_compressed_allocated":14336000,"data_compressed_original":229376000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16797696,"data_stored":268697624,"data_compressed":12759199,"data_compressed_allocated":16793600,"data_compressed_original":268697600,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16203776,"data_stored":259260416,"data_compressed":12311071,"data_compressed_allocated":16203776,"data_compressed_original":259260416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20869120,"data_stored":333840408,"data_compressed":15852527,"data_compressed_allocated":20865024,"data_compressed_original":333840384,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:34.987 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:13:35.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:35.145+0000 7fa514d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:35.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:35.155+0000 7fa514d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:35.649 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:35.649 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 74/16581 objects degraded (0.446%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":160,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":51,"num_osds":4,"num_up_osds":4,"osd_up_since":1726974750,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":24},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":25,"num_pools":3,"num_objects":5527,"data_bytes":362480203,"bytes_used":260542464,"bytes_avail":386286514176,"bytes_total":386547056640,"degraded_objects":74,"degraded_total":16581,"degraded_ratio":0.0044629395090766541,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":2736138,"recovering_keys_per_sec":0,"num_objects_recovered":176,"num_bytes_recovered":11534336,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":113},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:13:35.671 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:13:35.671 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:35.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:35.826+0000 7f3f9b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:35.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:35.837+0000 7f3f9b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:36.191 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:36.192 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:36.212 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-22T03:13:35.253569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328860235,"num_objects":5014,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":26281,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5974,"num_bytes_recovered":391512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26228,"ondisk_log_size":26228,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":256888,"kb_used_data":64248,"kb_used_omap":0,"kb_used_meta":192576,"kb_avail":377230472,"statfs":{"total":386547056640,"available":386284003328,"internally_reserved":0,"allocated":65789952,"data_stored":1025091576,"data_compressed":48652673,"data_compressed_allocated":64040960,"data_compressed_original":1024409600,"omap_allocated":0,"internal_metadata":197197824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-65011712,"num_objects":-992,"num_object_clones":0,"num_object_copies":-2976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":16384000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001150"},"pg_stats":[{"pgid":"3.f","version":"51'1708","reported_seq":2873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.213646+0000","last_change":"2024-09-22T03:13:30.111709+0000","last_active":"2024-09-22T03:13:32.213646+0000","last_peered":"2024-09-22T03:13:32.213646+0000","last_clean":"2024-09-22T03:13:32.213646+0000","last_became_active":"2024-09-22T03:13:30.105355+0000","last_became_peered":"2024-09-22T03:13:30.105355+0000","last_unstale":"2024-09-22T03:13:32.213646+0000","last_undegraded":"2024-09-22T03:13:32.213646+0000","last_fullsized":"2024-09-22T03:13:32.213646+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21954560,"num_objects":335,"num_object_clones":0,"num_object_copies":1005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1709,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":35061760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"51'1671","reported_seq":2847,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.180764+0000","last_change":"2024-09-22T03:13:27.683865+0000","last_active":"2024-09-22T03:13:33.180764+0000","last_peered":"2024-09-22T03:13:33.180764+0000","last_clean":"2024-09-22T03:13:33.180764+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:33.180764+0000","last_undegraded":"2024-09-22T03:13:33.180764+0000","last_fullsized":"2024-09-22T03:13:33.180764+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1671,"log_dups_size":0,"ondisk_log_size":1671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1672,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'1598","reported_seq":2660,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.280901+0000","last_change":"2024-09-22T03:13:29.087178+0000","last_active":"2024-09-22T03:13:30.280901+0000","last_peered":"2024-09-22T03:13:30.280901+0000","last_clean":"2024-09-22T03:13:30.280901+0000","last_became_active":"2024-09-22T03:13:29.071173+0000","last_became_peered":"2024-09-22T03:13:29.071173+0000","last_unstale":"2024-09-22T03:13:30.280901+0000","last_undegraded":"2024-09-22T03:13:30.280901+0000","last_fullsized":"2024-09-22T03:13:30.280901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1598,"log_dups_size":0,"ondisk_log_size":1598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364760,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1599,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":31916032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"51'1674","reported_seq":1789,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.181111+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:32.181111+0000","last_peered":"2024-09-22T03:13:32.181111+0000","last_clean":"2024-09-22T03:13:32.181111+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:32.181111+0000","last_undegraded":"2024-09-22T03:13:32.181111+0000","last_fullsized":"2024-09-22T03:13:32.181111+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1674,"log_dups_size":0,"ondisk_log_size":1674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1675,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"51'1652","reported_seq":1767,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.093486+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:33.093486+0000","last_peered":"2024-09-22T03:13:33.093486+0000","last_clean":"2024-09-22T03:13:33.093486+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:33.093486+0000","last_undegraded":"2024-09-22T03:13:33.093486+0000","last_fullsized":"2024-09-22T03:13:33.093486+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1652,"log_dups_size":0,"ondisk_log_size":1652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":299,"num_object_clones":0,"num_object_copies":897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1653,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"50'1604","reported_seq":2710,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060484+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:32.060484+0000","last_peered":"2024-09-22T03:13:32.060484+0000","last_clean":"2024-09-22T03:13:32.060484+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:32.060484+0000","last_undegraded":"2024-09-22T03:13:32.060484+0000","last_fullsized":"2024-09-22T03:13:32.060484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1604,"log_dups_size":0,"ondisk_log_size":1604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"51'1617","reported_seq":2647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.123645+0000","last_change":"2024-09-22T03:13:32.077980+0000","last_active":"2024-09-22T03:13:32.123645+0000","last_peered":"2024-09-22T03:13:32.123645+0000","last_clean":"2024-09-22T03:13:32.123645+0000","last_became_active":"2024-09-22T03:13:32.073222+0000","last_became_peered":"2024-09-22T03:13:32.073222+0000","last_unstale":"2024-09-22T03:13:32.123645+0000","last_undegraded":"2024-09-22T03:13:32.123645+0000","last_fullsized":"2024-09-22T03:13:32.123645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1617,"log_dups_size":0,"ondisk_log_size":1617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1618,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":467,"num_bytes_recovered":30605312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1641","reported_seq":2751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.259635+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:30.259635+0000","last_peered":"2024-09-22T03:13:30.259635+0000","last_clean":"2024-09-22T03:13:30.259635+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:30.259635+0000","last_undegraded":"2024-09-22T03:13:30.259635+0000","last_fullsized":"2024-09-22T03:13:30.259635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1641,"log_dups_size":0,"ondisk_log_size":1641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641536,"num_objects":376,"num_object_clones":0,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":127,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.058448+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:32.058448+0000","last_peered":"2024-09-22T03:13:32.058448+0000","last_clean":"2024-09-22T03:13:32.058448+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:32.058448+0000","last_undegraded":"2024-09-22T03:13:32.058448+0000","last_fullsized":"2024-09-22T03:13:32.058448+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"49'1645","reported_seq":2748,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.288834+0000","last_change":"2024-09-22T03:13:21.155496+0000","last_active":"2024-09-22T03:13:30.288834+0000","last_peered":"2024-09-22T03:13:30.288834+0000","last_clean":"2024-09-22T03:13:30.288834+0000","last_became_active":"2024-09-22T03:13:21.077994+0000","last_became_peered":"2024-09-22T03:13:21.077994+0000","last_unstale":"2024-09-22T03:13:30.288834+0000","last_undegraded":"2024-09-22T03:13:30.288834+0000","last_fullsized":"2024-09-22T03:13:30.288834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1645,"log_dups_size":0,"ondisk_log_size":1645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1646,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.057686+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:32.057686+0000","last_peered":"2024-09-22T03:13:32.057686+0000","last_clean":"2024-09-22T03:13:32.057686+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:32.057686+0000","last_undegraded":"2024-09-22T03:13:32.057686+0000","last_fullsized":"2024-09-22T03:13:32.057686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"51'1619","reported_seq":1734,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.249583+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:33.249583+0000","last_peered":"2024-09-22T03:13:33.249583+0000","last_clean":"2024-09-22T03:13:33.249583+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:33.249583+0000","last_undegraded":"2024-09-22T03:13:33.249583+0000","last_fullsized":"2024-09-22T03:13:33.249583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1619,"log_dups_size":0,"ondisk_log_size":1619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":282,"num_object_clones":0,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1620,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.057566+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:32.057566+0000","last_peered":"2024-09-22T03:13:32.057566+0000","last_clean":"2024-09-22T03:13:32.057566+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:32.057566+0000","last_undegraded":"2024-09-22T03:13:32.057566+0000","last_fullsized":"2024-09-22T03:13:32.057566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"51'1667","reported_seq":2756,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.222149+0000","last_change":"2024-09-22T03:13:24.076522+0000","last_active":"2024-09-22T03:13:33.222149+0000","last_peered":"2024-09-22T03:13:33.222149+0000","last_clean":"2024-09-22T03:13:33.222149+0000","last_became_active":"2024-09-22T03:13:24.069253+0000","last_became_peered":"2024-09-22T03:13:24.069253+0000","last_unstale":"2024-09-22T03:13:33.222149+0000","last_undegraded":"2024-09-22T03:13:33.222149+0000","last_fullsized":"2024-09-22T03:13:33.222149+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1667,"log_dups_size":0,"ondisk_log_size":1667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1668,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":135,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.064137+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:32.064137+0000","last_peered":"2024-09-22T03:13:32.064137+0000","last_clean":"2024-09-22T03:13:32.064137+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:32.064137+0000","last_undegraded":"2024-09-22T03:13:32.064137+0000","last_fullsized":"2024-09-22T03:13:32.064137+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":258,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060174+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:32.060174+0000","last_peered":"2024-09-22T03:13:32.060174+0000","last_clean":"2024-09-22T03:13:32.060174+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:32.060174+0000","last_undegraded":"2024-09-22T03:13:32.060174+0000","last_fullsized":"2024-09-22T03:13:32.060174+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"51'1603","reported_seq":2640,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.142658+0000","last_change":"2024-09-22T03:13:32.090877+0000","last_active":"2024-09-22T03:13:33.142658+0000","last_peered":"2024-09-22T03:13:33.142658+0000","last_clean":"2024-09-22T03:13:33.142658+0000","last_became_active":"2024-09-22T03:13:32.076387+0000","last_became_peered":"2024-09-22T03:13:32.076387+0000","last_unstale":"2024-09-22T03:13:33.142658+0000","last_undegraded":"2024-09-22T03:13:33.142658+0000","last_fullsized":"2024-09-22T03:13:33.142658+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1603,"log_dups_size":0,"ondisk_log_size":1603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709376,"num_objects":316,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":30932992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"51'1620","reported_seq":2727,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.248994+0000","last_change":"2024-09-22T03:13:27.069795+0000","last_active":"2024-09-22T03:13:33.248994+0000","last_peered":"2024-09-22T03:13:33.248994+0000","last_clean":"2024-09-22T03:13:33.248994+0000","last_became_active":"2024-09-22T03:13:27.064201+0000","last_became_peered":"2024-09-22T03:13:27.064201+0000","last_unstale":"2024-09-22T03:13:33.248994+0000","last_undegraded":"2024-09-22T03:13:33.248994+0000","last_fullsized":"2024-09-22T03:13:33.248994+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1620,"log_dups_size":0,"ondisk_log_size":1620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1621,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":32964608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.061459+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:30.061459+0000","last_peered":"2024-09-22T03:13:30.061459+0000","last_clean":"2024-09-22T03:13:30.061459+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:30.061459+0000","last_undegraded":"2024-09-22T03:13:30.061459+0000","last_fullsized":"2024-09-22T03:13:30.061459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"51'1591","reported_seq":2662,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.163506+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:33.163506+0000","last_peered":"2024-09-22T03:13:33.163506+0000","last_clean":"2024-09-22T03:13:33.163506+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:33.163506+0000","last_undegraded":"2024-09-22T03:13:33.163506+0000","last_fullsized":"2024-09-22T03:13:33.163506+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1591,"log_dups_size":0,"ondisk_log_size":1591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18219008,"num_objects":278,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1592,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":127,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060264+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:32.060264+0000","last_peered":"2024-09-22T03:13:32.060264+0000","last_clean":"2024-09-22T03:13:32.060264+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:32.060264+0000","last_undegraded":"2024-09-22T03:13:32.060264+0000","last_fullsized":"2024-09-22T03:13:32.060264+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"51'1614","reported_seq":1729,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:33.197162+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:33.197162+0000","last_peered":"2024-09-22T03:13:33.197162+0000","last_clean":"2024-09-22T03:13:33.197162+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:33.197162+0000","last_undegraded":"2024-09-22T03:13:33.197162+0000","last_fullsized":"2024-09-22T03:13:33.197162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1614,"log_dups_size":0,"ondisk_log_size":1614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18546688,"num_objects":283,"num_object_clones":0,"num_object_copies":849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1615,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.060862+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:30.060862+0000","last_peered":"2024-09-22T03:13:30.060862+0000","last_clean":"2024-09-22T03:13:30.060862+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:30.060862+0000","last_undegraded":"2024-09-22T03:13:30.060862+0000","last_fullsized":"2024-09-22T03:13:30.060862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"48'1604","reported_seq":2697,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.056381+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:30.056381+0000","last_peered":"2024-09-22T03:13:30.056381+0000","last_clean":"2024-09-22T03:13:30.056381+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:30.056381+0000","last_undegraded":"2024-09-22T03:13:30.056381+0000","last_fullsized":"2024-09-22T03:13:30.056381+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1604,"log_dups_size":0,"ondisk_log_size":1604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-22T03:13:30.061228+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:30.061228+0000","last_peered":"2024-09-22T03:13:30.061228+0000","last_clean":"2024-09-22T03:13:30.061228+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:30.061228+0000","last_undegraded":"2024-09-22T03:13:30.061228+0000","last_fullsized":"2024-09-22T03:13:30.061228+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":328269848,"num_objects":5010,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":26142,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5974,"num_bytes_recovered":391512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61526016,"data_stored":984219720,"data_compressed":46736013,"data_compressed_allocated":61513728,"data_compressed_original":984219648,"omap_allocated":0,"internal_metadata":0},"log_size":26128,"ondisk_log_size":26128,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673109,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67232,"kb_used_data":18192,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94304608,"statfs":{"total":96636764160,"available":96567918592,"internally_reserved":0,"allocated":18628608,"data_stored":291336728,"data_compressed":13826615,"data_compressed_allocated":18198528,"data_compressed_original":291176448,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673109,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63412,"kb_used_data":14372,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94308428,"statfs":{"total":96636764160,"available":96571830272,"internally_reserved":0,"allocated":14716928,"data_stored":228422688,"data_compressed":10841059,"data_compressed_allocated":14270464,"data_compressed_original":228245504,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149756,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66732,"kb_used_data":17052,"kb_used_omap":0,"kb_used_meta":49664,"kb_avail":94305108,"statfs":{"total":96636764160,"available":96568430592,"internally_reserved":0,"allocated":17461248,"data_stored":272521836,"data_compressed":12935435,"data_compressed_allocated":17027072,"data_compressed_original":272351232,"omap_allocated":0,"internal_metadata":50855936},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59512,"kb_used_data":14632,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94312328,"statfs":{"total":96636764160,"available":96575823872,"internally_reserved":0,"allocated":14983168,"data_stored":232810324,"data_compressed":11049564,"data_compressed_allocated":14544896,"data_compressed_original":232636416,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14340096,"data_stored":229376024,"data_compressed":10892000,"data_compressed_allocated":14336000,"data_compressed_original":229376000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16797696,"data_stored":268697624,"data_compressed":12759199,"data_compressed_allocated":16793600,"data_compressed_original":268697600,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13332480,"data_stored":213319680,"data_compressed":10129559,"data_compressed_allocated":13332480,"data_compressed_original":213319680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17055744,"data_stored":272826392,"data_compressed":12955255,"data_compressed_allocated":17051648,"data_compressed_original":272826368,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:36.903 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:37.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:37.058+0000 7faa7106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:37.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:37.065+0000 7faa7106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:39.214 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:13:39.308 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:13:39.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:39.355+0000 7f1fa8f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:39.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:39.370+0000 7f1fa8f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:39.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:39.464+0000 7fa98cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:39.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:39.473+0000 7fa98cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:39.699 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:13:39.700 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:13:39.719 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-22T03:13:39.254292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225182283,"num_objects":3432,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":27863,"num_write_kb":998697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5974,"num_bytes_recovered":391512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27808,"ondisk_log_size":27808,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":255004,"kb_used_data":45716,"kb_used_omap":0,"kb_used_meta":209216,"kb_avail":377232356,"statfs":{"total":386547056640,"available":386285932544,"internally_reserved":0,"allocated":46813184,"data_stored":721079957,"data_compressed":34216108,"data_compressed_allocated":45039616,"data_compressed_original":720388096,"omap_allocated":0,"internal_metadata":214237184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-152240128,"num_objects":-2323,"num_object_clones":0,"num_object_copies":-6969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-241,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2323,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":15794176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.205055"},"pg_stats":[{"pgid":"3.f","version":"51'1828","reported_seq":2993,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:37.204047+0000","last_change":"2024-09-22T03:13:30.111709+0000","last_active":"2024-09-22T03:13:37.204047+0000","last_peered":"2024-09-22T03:13:37.204047+0000","last_clean":"2024-09-22T03:13:37.204047+0000","last_became_active":"2024-09-22T03:13:30.105355+0000","last_became_peered":"2024-09-22T03:13:30.105355+0000","last_unstale":"2024-09-22T03:13:37.204047+0000","last_undegraded":"2024-09-22T03:13:37.204047+0000","last_fullsized":"2024-09-22T03:13:37.204047+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1828,"log_dups_size":0,"ondisk_log_size":1828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14090240,"num_objects":215,"num_object_clones":0,"num_object_copies":645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1829,"num_write_kb":65408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":35061760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"51'1788","reported_seq":2964,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.113783+0000","last_change":"2024-09-22T03:13:27.683865+0000","last_active":"2024-09-22T03:13:38.113783+0000","last_peered":"2024-09-22T03:13:38.113783+0000","last_clean":"2024-09-22T03:13:38.113783+0000","last_became_active":"2024-09-22T03:13:18.109457+0000","last_became_peered":"2024-09-22T03:13:18.109457+0000","last_unstale":"2024-09-22T03:13:38.113783+0000","last_undegraded":"2024-09-22T03:13:38.113783+0000","last_fullsized":"2024-09-22T03:13:38.113783+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1788,"log_dups_size":0,"ondisk_log_size":1788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11468800,"num_objects":175,"num_object_clones":0,"num_object_copies":525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1789,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"51'1683","reported_seq":2751,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:35.274677+0000","last_change":"2024-09-22T03:13:29.087178+0000","last_active":"2024-09-22T03:13:35.274677+0000","last_peered":"2024-09-22T03:13:35.274677+0000","last_clean":"2024-09-22T03:13:35.274677+0000","last_became_active":"2024-09-22T03:13:29.071173+0000","last_became_peered":"2024-09-22T03:13:29.071173+0000","last_unstale":"2024-09-22T03:13:35.274677+0000","last_undegraded":"2024-09-22T03:13:35.274677+0000","last_fullsized":"2024-09-22T03:13:35.274677+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1683,"log_dups_size":0,"ondisk_log_size":1683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15794200,"num_objects":242,"num_object_clones":0,"num_object_copies":726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1684,"num_write_kb":61569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":31916032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"51'1791","reported_seq":1906,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:37.191937+0000","last_change":"2024-09-22T03:11:14.443063+0000","last_active":"2024-09-22T03:13:37.191937+0000","last_peered":"2024-09-22T03:13:37.191937+0000","last_clean":"2024-09-22T03:13:37.191937+0000","last_became_active":"2024-09-22T03:11:14.440305+0000","last_became_peered":"2024-09-22T03:11:14.440305+0000","last_unstale":"2024-09-22T03:13:37.191937+0000","last_undegraded":"2024-09-22T03:13:37.191937+0000","last_fullsized":"2024-09-22T03:13:37.191937+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1791,"log_dups_size":0,"ondisk_log_size":1791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":206,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1792,"num_write_kb":63936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"51'1746","reported_seq":1861,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.128263+0000","last_change":"2024-09-22T03:11:14.438597+0000","last_active":"2024-09-22T03:13:38.128263+0000","last_peered":"2024-09-22T03:13:38.128263+0000","last_clean":"2024-09-22T03:13:38.128263+0000","last_became_active":"2024-09-22T03:11:14.430836+0000","last_became_peered":"2024-09-22T03:11:14.430836+0000","last_unstale":"2024-09-22T03:13:38.128263+0000","last_undegraded":"2024-09-22T03:13:38.128263+0000","last_fullsized":"2024-09-22T03:13:38.128263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1746,"log_dups_size":0,"ondisk_log_size":1746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13434880,"num_objects":205,"num_object_clones":0,"num_object_copies":615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1747,"num_write_kb":62464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"51'1705","reported_seq":2812,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:37.123207+0000","last_change":"2024-09-22T03:12:48.385492+0000","last_active":"2024-09-22T03:13:37.123207+0000","last_peered":"2024-09-22T03:13:37.123207+0000","last_clean":"2024-09-22T03:13:37.123207+0000","last_became_active":"2024-09-22T03:12:48.377494+0000","last_became_peered":"2024-09-22T03:12:48.377494+0000","last_unstale":"2024-09-22T03:13:37.123207+0000","last_undegraded":"2024-09-22T03:13:37.123207+0000","last_fullsized":"2024-09-22T03:13:37.123207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1705,"log_dups_size":0,"ondisk_log_size":1705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14548992,"num_objects":222,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1706,"num_write_kb":61696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":32571392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"51'1706","reported_seq":2736,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:37.162669+0000","last_change":"2024-09-22T03:13:32.077980+0000","last_active":"2024-09-22T03:13:37.162669+0000","last_peered":"2024-09-22T03:13:37.162669+0000","last_clean":"2024-09-22T03:13:37.162669+0000","last_became_active":"2024-09-22T03:13:32.073222+0000","last_became_peered":"2024-09-22T03:13:32.073222+0000","last_unstale":"2024-09-22T03:13:37.162669+0000","last_undegraded":"2024-09-22T03:13:37.162669+0000","last_fullsized":"2024-09-22T03:13:37.162669+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1706,"log_dups_size":0,"ondisk_log_size":1706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":219,"num_object_clones":0,"num_object_copies":657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1707,"num_write_kb":61632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":467,"num_bytes_recovered":30605312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"51'1728","reported_seq":2844,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:35.162808+0000","last_change":"2024-09-22T03:13:06.074368+0000","last_active":"2024-09-22T03:13:35.162808+0000","last_peered":"2024-09-22T03:13:35.162808+0000","last_clean":"2024-09-22T03:13:35.162808+0000","last_became_active":"2024-09-22T03:13:06.063920+0000","last_became_peered":"2024-09-22T03:13:06.063920+0000","last_unstale":"2024-09-22T03:13:35.162808+0000","last_undegraded":"2024-09-22T03:13:35.162808+0000","last_fullsized":"2024-09-22T03:13:35.162808+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1728,"log_dups_size":0,"ondisk_log_size":1728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":289,"num_object_clones":0,"num_object_copies":867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1729,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":127,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.058448+0000","last_change":"2024-09-22T03:11:08.277835+0000","last_active":"2024-09-22T03:13:32.058448+0000","last_peered":"2024-09-22T03:13:32.058448+0000","last_clean":"2024-09-22T03:13:32.058448+0000","last_became_active":"2024-09-22T03:11:06.329113+0000","last_became_peered":"2024-09-22T03:11:06.329113+0000","last_unstale":"2024-09-22T03:13:32.058448+0000","last_undegraded":"2024-09-22T03:13:32.058448+0000","last_fullsized":"2024-09-22T03:13:32.058448+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"51'1736","reported_seq":2845,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:35.242235+0000","last_change":"2024-09-22T03:13:21.155496+0000","last_active":"2024-09-22T03:13:35.242235+0000","last_peered":"2024-09-22T03:13:35.242235+0000","last_clean":"2024-09-22T03:13:35.242235+0000","last_became_active":"2024-09-22T03:13:21.077994+0000","last_became_peered":"2024-09-22T03:13:21.077994+0000","last_unstale":"2024-09-22T03:13:35.242235+0000","last_undegraded":"2024-09-22T03:13:35.242235+0000","last_fullsized":"2024-09-22T03:13:35.242235+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1736,"log_dups_size":0,"ondisk_log_size":1736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":253,"num_object_clones":0,"num_object_copies":759,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1737,"num_write_kb":63680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":32833536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.057686+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:13:32.057686+0000","last_peered":"2024-09-22T03:13:32.057686+0000","last_clean":"2024-09-22T03:13:32.057686+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:13:32.057686+0000","last_undegraded":"2024-09-22T03:13:32.057686+0000","last_fullsized":"2024-09-22T03:13:32.057686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"51'1726","reported_seq":1841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.073257+0000","last_change":"2024-09-22T03:11:14.442397+0000","last_active":"2024-09-22T03:13:38.073257+0000","last_peered":"2024-09-22T03:13:38.073257+0000","last_clean":"2024-09-22T03:13:38.073257+0000","last_became_active":"2024-09-22T03:11:14.438948+0000","last_became_peered":"2024-09-22T03:11:14.438948+0000","last_unstale":"2024-09-22T03:13:38.073257+0000","last_undegraded":"2024-09-22T03:13:38.073257+0000","last_fullsized":"2024-09-22T03:13:38.073257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1726,"log_dups_size":0,"ondisk_log_size":1726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11468800,"num_objects":175,"num_object_clones":0,"num_object_copies":525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1727,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.057566+0000","last_change":"2024-09-22T03:12:32.056191+0000","last_active":"2024-09-22T03:13:32.057566+0000","last_peered":"2024-09-22T03:13:32.057566+0000","last_clean":"2024-09-22T03:13:32.057566+0000","last_became_active":"2024-09-22T03:12:31.085427+0000","last_became_peered":"2024-09-22T03:12:31.085427+0000","last_unstale":"2024-09-22T03:13:32.057566+0000","last_undegraded":"2024-09-22T03:13:32.057566+0000","last_fullsized":"2024-09-22T03:13:32.057566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"51'1754","reported_seq":2843,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.206526+0000","last_change":"2024-09-22T03:13:24.076522+0000","last_active":"2024-09-22T03:13:38.206526+0000","last_peered":"2024-09-22T03:13:38.206526+0000","last_clean":"2024-09-22T03:13:38.206526+0000","last_became_active":"2024-09-22T03:13:24.069253+0000","last_became_peered":"2024-09-22T03:13:24.069253+0000","last_unstale":"2024-09-22T03:13:38.206526+0000","last_undegraded":"2024-09-22T03:13:38.206526+0000","last_fullsized":"2024-09-22T03:13:38.206526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1754,"log_dups_size":0,"ondisk_log_size":1754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":201,"num_object_clones":0,"num_object_copies":603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":32309248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":135,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.064137+0000","last_change":"2024-09-22T03:12:32.374096+0000","last_active":"2024-09-22T03:13:32.064137+0000","last_peered":"2024-09-22T03:13:32.064137+0000","last_clean":"2024-09-22T03:13:32.064137+0000","last_became_active":"2024-09-22T03:12:31.105678+0000","last_became_peered":"2024-09-22T03:12:31.105678+0000","last_unstale":"2024-09-22T03:13:32.064137+0000","last_undegraded":"2024-09-22T03:13:32.064137+0000","last_fullsized":"2024-09-22T03:13:32.064137+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":258,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060174+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:13:32.060174+0000","last_peered":"2024-09-22T03:13:32.060174+0000","last_clean":"2024-09-22T03:13:32.060174+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:13:32.060174+0000","last_undegraded":"2024-09-22T03:13:32.060174+0000","last_fullsized":"2024-09-22T03:13:32.060174+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"51'1705","reported_seq":2742,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.215040+0000","last_change":"2024-09-22T03:13:32.090877+0000","last_active":"2024-09-22T03:13:38.215040+0000","last_peered":"2024-09-22T03:13:38.215040+0000","last_clean":"2024-09-22T03:13:38.215040+0000","last_became_active":"2024-09-22T03:13:32.076387+0000","last_became_peered":"2024-09-22T03:13:32.076387+0000","last_unstale":"2024-09-22T03:13:38.215040+0000","last_undegraded":"2024-09-22T03:13:38.215040+0000","last_fullsized":"2024-09-22T03:13:38.215040+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1705,"log_dups_size":0,"ondisk_log_size":1705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14024704,"num_objects":214,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1706,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":30932992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"51'1721","reported_seq":2828,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.181895+0000","last_change":"2024-09-22T03:13:27.069795+0000","last_active":"2024-09-22T03:13:38.181895+0000","last_peered":"2024-09-22T03:13:38.181895+0000","last_clean":"2024-09-22T03:13:38.181895+0000","last_became_active":"2024-09-22T03:13:27.064201+0000","last_became_peered":"2024-09-22T03:13:27.064201+0000","last_unstale":"2024-09-22T03:13:38.181895+0000","last_undegraded":"2024-09-22T03:13:38.181895+0000","last_fullsized":"2024-09-22T03:13:38.181895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1721,"log_dups_size":0,"ondisk_log_size":1721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12451840,"num_objects":190,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1722,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":32964608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":102,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.062390+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:13:32.062390+0000","last_peered":"2024-09-22T03:13:32.062390+0000","last_clean":"2024-09-22T03:13:32.062390+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:13:32.062390+0000","last_undegraded":"2024-09-22T03:13:32.062390+0000","last_fullsized":"2024-09-22T03:13:32.062390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"51'1688","reported_seq":2759,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.141564+0000","last_change":"2024-09-22T03:13:16.093536+0000","last_active":"2024-09-22T03:13:38.141564+0000","last_peered":"2024-09-22T03:13:38.141564+0000","last_clean":"2024-09-22T03:13:38.141564+0000","last_became_active":"2024-09-22T03:13:16.086050+0000","last_became_peered":"2024-09-22T03:13:16.086050+0000","last_unstale":"2024-09-22T03:13:38.141564+0000","last_undegraded":"2024-09-22T03:13:38.141564+0000","last_fullsized":"2024-09-22T03:13:38.141564+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1688,"log_dups_size":0,"ondisk_log_size":1688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11862016,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1689,"num_write_kb":59840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":127,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060264+0000","last_change":"2024-09-22T03:11:08.280234+0000","last_active":"2024-09-22T03:13:32.060264+0000","last_peered":"2024-09-22T03:13:32.060264+0000","last_clean":"2024-09-22T03:13:32.060264+0000","last_became_active":"2024-09-22T03:11:06.309381+0000","last_became_peered":"2024-09-22T03:11:06.309381+0000","last_unstale":"2024-09-22T03:13:32.060264+0000","last_undegraded":"2024-09-22T03:13:32.060264+0000","last_fullsized":"2024-09-22T03:13:32.060264+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"51'1710","reported_seq":1825,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:38.068665+0000","last_change":"2024-09-22T03:11:14.435545+0000","last_active":"2024-09-22T03:13:38.068665+0000","last_peered":"2024-09-22T03:13:38.068665+0000","last_clean":"2024-09-22T03:13:38.068665+0000","last_became_active":"2024-09-22T03:11:14.430915+0000","last_became_peered":"2024-09-22T03:11:14.430915+0000","last_unstale":"2024-09-22T03:13:38.068665+0000","last_undegraded":"2024-09-22T03:13:38.068665+0000","last_fullsized":"2024-09-22T03:13:38.068665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1710,"log_dups_size":0,"ondisk_log_size":1710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12255232,"num_objects":187,"num_object_clones":0,"num_object_copies":561,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1711,"num_write_kb":60736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":102,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.060658+0000","last_change":"2024-09-22T03:12:32.030907+0000","last_active":"2024-09-22T03:13:32.060658+0000","last_peered":"2024-09-22T03:13:32.060658+0000","last_clean":"2024-09-22T03:13:32.060658+0000","last_became_active":"2024-09-22T03:12:31.119375+0000","last_became_peered":"2024-09-22T03:12:31.119375+0000","last_unstale":"2024-09-22T03:13:32.060658+0000","last_undegraded":"2024-09-22T03:13:32.060658+0000","last_fullsized":"2024-09-22T03:13:32.060658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"51'1693","reported_seq":2793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:35.175855+0000","last_change":"2024-09-22T03:13:04.455751+0000","last_active":"2024-09-22T03:13:35.175855+0000","last_peered":"2024-09-22T03:13:35.175855+0000","last_clean":"2024-09-22T03:13:35.175855+0000","last_became_active":"2024-09-22T03:13:04.448017+0000","last_became_peered":"2024-09-22T03:13:04.448017+0000","last_unstale":"2024-09-22T03:13:35.175855+0000","last_undegraded":"2024-09-22T03:13:35.175855+0000","last_fullsized":"2024-09-22T03:13:35.175855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:13.388416+0000","last_clean_scrub_stamp":"2024-09-22T03:11:13.388416+0000","objects_scrubbed":0,"log_size":1693,"log_dups_size":0,"ondisk_log_size":1693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":254,"num_object_clones":0,"num_object_copies":762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1694,"num_write_kb":62336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":32505856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:13:32.061020+0000","last_change":"2024-09-22T03:12:32.031005+0000","last_active":"2024-09-22T03:13:32.061020+0000","last_peered":"2024-09-22T03:13:32.061020+0000","last_clean":"2024-09-22T03:13:32.061020+0000","last_became_active":"2024-09-22T03:12:31.127444+0000","last_became_peered":"2024-09-22T03:12:31.127444+0000","last_unstale":"2024-09-22T03:13:32.061020+0000","last_undegraded":"2024-09-22T03:13:32.061020+0000","last_fullsized":"2024-09-22T03:13:32.061020+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":224591896,"num_objects":3428,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27724,"num_write_kb":996801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5974,"num_bytes_recovered":391512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41832448,"data_stored":669122632,"data_compressed":31773520,"data_compressed_allocated":41820160,"data_compressed_original":669122560,"omap_allocated":0,"internal_metadata":0},"log_size":27708,"ondisk_log_size":27708,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65448,"kb_used_data":12244,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94306392,"statfs":{"total":96636764160,"available":96569745408,"internally_reserved":0,"allocated":12537856,"data_stored":193884696,"data_compressed":9199072,"data_compressed_allocated":12107776,"data_compressed_original":193724416,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63004,"kb_used_data":9800,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308836,"statfs":{"total":96636764160,"available":96572248064,"internally_reserved":0,"allocated":10035200,"data_stored":153515040,"data_compressed":7284044,"data_compressed_allocated":9588736,"data_compressed_original":153337856,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149761,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66780,"kb_used_data":12940,"kb_used_omap":0,"kb_used_meta":53824,"kb_avail":94305060,"statfs":{"total":96636764160,"available":96568381440,"internally_reserved":0,"allocated":13250560,"data_stored":204895288,"data_compressed":9723852,"data_compressed_allocated":12800000,"data_compressed_original":204718080,"omap_allocated":0,"internal_metadata":55115776},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59772,"kb_used_data":10732,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94312068,"statfs":{"total":96636764160,"available":96575557632,"internally_reserved":0,"allocated":10989568,"data_stored":168784933,"data_compressed":8009140,"data_compressed_allocated":10543104,"data_compressed_original":168607744,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9605120,"data_stored":153616408,"data_compressed":7294528,"data_compressed_allocated":9601024,"data_compressed_original":153616384,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12562432,"data_stored":200933400,"data_compressed":9541392,"data_compressed_allocated":12558336,"data_compressed_original":200933376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8663040,"data_stored":138608640,"data_compressed":6581880,"data_compressed_allocated":8663040,"data_compressed_original":138608640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11001856,"data_stored":175964184,"data_compressed":8355720,"data_compressed_allocated":10997760,"data_compressed_original":175964160,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:13:39.720 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:13:39.720 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:13:39.720 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 2, 3, 1] 2024-09-22T03:13:39.721 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T03:13:39.721 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:13:41.752 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:13:41.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:41.865+0000 7f6a17ca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:41.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:13:41.869+0000 7f6a17ca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:13:44.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T03:13:45.822 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T03:13:50.326 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T03:13:50.824 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-22T03:13:50.824 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:13:50.824 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-22T03:13:50.824 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:13:50.824 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:13:50.827 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:13:50.827 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:13:50.937 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:13:50.940 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:13:50.941 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:13:55.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:13:56.038 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:13:56.040 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:13:56.041 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:14:01.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:14:01.138 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:14:01.140 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:01.141 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:14:05.031 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:14:05.036 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:14:05.036 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:14:06.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:14:06.245 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:14:06.245 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [ 2024-09-22T03:14:06.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 47, 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.070383+0000" 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.070390+0000" 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.072384+0000" 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.072973+0000" 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073867+0000" 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.074798+0000" 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.075568+0000" 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:14:06.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.075666+0000", 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "blocker": { 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "op_type": "PGActivationBlocker", 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "pgid": "3.1" 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 51, 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073617+0000" 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073622+0000" 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.074247+0000" 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.074840+0000" 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.075918+0000" 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.076595+0000" 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.076783+0000", 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "blocker": { 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: "waiting": 1, 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "held_by_operation_id": 47 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:14:06.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.267 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 40, 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:14:06.270 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.051519+0000" 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.051564+0000" 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.052933+0000" 2024-09-22T03:14:06.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.053245+0000" 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.071858+0000" 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.072571+0000" 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073014+0000" 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073393+0000" 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073610+0000", 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "blocker": { 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "op_type": "PGActivationBlocker", 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "pgid": "3.b" 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.277 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.282 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 45, 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.068553+0000" 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.068561+0000" 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.069512+0000" 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.072294+0000" 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073125+0000" 2024-09-22T03:14:06.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.073919+0000" 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:14:06.074101+0000", 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "blocker": { 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "waiting": 1, 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "held_by_operation_id": 40 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:14:06.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:14:06.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:14:06.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-22T03:14:06.294 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:14:07.037 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:07.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:07.181+0000 7fef91557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:07.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:07.188+0000 7fef91557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:09.464 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:09.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:09.605+0000 7fc37b86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:09.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:09.612+0000 7fc37b86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.296 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:11.296 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:11.297 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:14:11.297 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:14:11.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.418+0000 7f3eae317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.430+0000 7f3eae317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.462 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:11.463 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:11.467 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:11.468 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:11.468 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:14:11.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.605+0000 7ffaff4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.617+0000 7ffaff4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:11.642 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:11.643 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:11.643 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:11.643 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:11.646 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:11.646 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:11.647 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:14:11.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.788+0000 7f32a36d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.802+0000 7f32a36d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.824 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:11.824 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:11.824 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:11.824 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:11.824 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:11.825 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:11.825 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:11.825 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:11.825 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:11.825 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:11.825 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:11.825 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:11.829 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:11.830 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:11.830 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:14:11.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.974+0000 7f4587c23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:11.988+0000 7f4587c23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:12.029 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:12.137 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:12.137 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:12.138 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:12.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:12.144 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:12.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:12.174+0000 7f3b4026c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:12.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:12.182+0000 7f3b4026c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:14.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:14.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:14.687+0000 7f75b422c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:14.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:14.693+0000 7f75b422c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:15.244 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Removed 15575 objects 2024-09-22T03:14:15.244 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Clean up completed and total clean up time :88.7379 2024-09-22T03:14:16.928 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:17.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.035+0000 7fbe7bd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.040+0000 7fbe7bd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.145 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:17.145 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:17.145 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:14:17.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:14:17.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.244+0000 7fbe752fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.252+0000 7fbe752fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:17.268 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:17.268 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:17.268 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:14:17.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.367+0000 7f5463f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.375+0000 7f5463f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.388 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:17.388 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:17.389 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:17.392 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:17.392 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:17.393 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:14:17.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.491+0000 7f67788ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.498+0000 7f67788ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.512 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:17.512 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:17.513 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:17.515 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:17.516 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:17.516 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:14:17.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.613+0000 7f2e90557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:17.621+0000 7f2e90557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:17.637 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:17.639 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:17.640 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:18.498 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-22T03:14:18.498 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-22T03:14:18.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:18.599+0000 7f27c1ca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:18.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:18.610+0000 7f27c1ca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:19.110 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_0' removed 2024-09-22T03:14:19.112 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:19.138 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:14:19.139 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:14:19.139 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-22T03:14:19.139 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_1 16 2024-09-22T03:14:19.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:19.240+0000 7fb543d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:19.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:19.246+0000 7fb543d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:19.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:19.321+0000 7efd10397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:19.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:19.332+0000 7efd10397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:20.114 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_1' created 2024-09-22T03:14:20.133 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-22T03:14:20.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:20.247+0000 7fd60d82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:20.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:20.259+0000 7fd60d82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:21.117 INFO:teuthology.orchestra.run.smithi146.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-22T03:14:21.393 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:21.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:21.492+0000 7fd35b658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:21.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:21.497+0000 7fd35b658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:22.141 DEBUG:teuthology.orchestra.run.smithi146:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-22T03:14:22.144 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:14:22.144 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:14:22.200 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:hints = 1 2024-09-22T03:14:22.200 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:14:22.200 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object prefix: benchmark_data_smithi146_35946 2024-09-22T03:14:22.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:14:22.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:14:22.640 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:22.640 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:22.641 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:14:22.641 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:14:22.804 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:22.795+0000 7f9724a63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:22.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:22.806+0000 7f9724a63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:22.835 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:22.835 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:22.836 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:22.841 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:22.841 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:22.841 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:14:23.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.000+0000 7f8c3c997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:23.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.017+0000 7f8c3759e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:23.055 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:23.055 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:23.056 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:23.061 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:23.061 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:23.061 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:14:23.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 1 16 410 394 24.6236 24.625 0.0744122 0.0395165 2024-09-22T03:14:23.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.213+0000 7f762e46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:23.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.229+0000 7f762e46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:23.256 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:23.260 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:23.261 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:23.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:14:23.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.421+0000 7f0ebed97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:23.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.436+0000 7f0ebed97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:23.461 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:23.462 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:23.462 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:23.462 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:23.462 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:23.462 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:23.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:23.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:23.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:23.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:23.463 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:23.463 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:23.468 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:23.468 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:23.565 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:23.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.715+0000 7f0866997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:23.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:23.722+0000 7f0866997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:24.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 2 16 827 811 25.3419 26.0625 0.0291477 0.0391198 2024-09-22T03:14:25.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 3 16 1248 1232 25.6647 26.3125 0.0402421 0.0387193 2024-09-22T03:14:25.900 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:26.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:26.051+0000 7fcbe8797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:26.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:26.059+0000 7fcbe8797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:26.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 4 16 1672 1656 25.873 26.5 0.0268055 0.0384049 2024-09-22T03:14:27.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 5 16 2103 2087 26.0854 26.9375 0.040713 0.0380996 2024-09-22T03:14:28.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 6 16 2539 2523 26.2791 27.25 0.0354003 0.0379259 2024-09-22T03:14:28.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:28.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:28.408+0000 7f26db797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:28.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:28.418+0000 7f26db797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:28.469 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:28.469 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:28.469 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:14:28.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:28.624+0000 7fcbb224f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:28.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:28.638+0000 7fcbb224f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:28.942 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:14:28.942 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:14:28.961 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-22T03:14:27.263213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51970611,"num_objects":788,"num_object_clones":0,"num_object_copies":2364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":923,"num_write_kb":52072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":210456,"kb_used_data":10068,"kb_used_omap":0,"kb_used_meta":200320,"kb_avail":377276904,"statfs":{"total":386547056640,"available":386331549696,"internally_reserved":0,"allocated":10309632,"data_stored":133515144,"data_compressed":6308890,"data_compressed_allocated":8310784,"data_compressed_original":132726784,"omap_allocated":0,"internal_metadata":205127680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51380224,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":50176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.149630"},"pg_stats":[{"pgid":"4.f","version":"58'31","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:23.200246+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:14:23.200246+0000","last_peered":"2024-09-22T03:14:23.200246+0000","last_clean":"2024-09-22T03:14:23.200246+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:14:23.200246+0000","last_undegraded":"2024-09-22T03:14:23.200246+0000","last_fullsized":"2024-09-22T03:14:23.200246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"58'21","reported_seq":34,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:23.208514+0000","last_change":"2024-09-22T03:14:21.195156+0000","last_active":"2024-09-22T03:14:23.208514+0000","last_peered":"2024-09-22T03:14:23.208514+0000","last_clean":"2024-09-22T03:14:23.208514+0000","last_became_active":"2024-09-22T03:14:21.188092+0000","last_became_peered":"2024-09-22T03:14:21.188092+0000","last_unstale":"2024-09-22T03:14:23.208514+0000","last_undegraded":"2024-09-22T03:14:23.208514+0000","last_fullsized":"2024-09-22T03:14:23.208514+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"58'85","reported_seq":98,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:25.290105+0000","last_change":"2024-09-22T03:14:21.204545+0000","last_active":"2024-09-22T03:14:25.290105+0000","last_peered":"2024-09-22T03:14:25.290105+0000","last_clean":"2024-09-22T03:14:25.290105+0000","last_became_active":"2024-09-22T03:14:21.197364+0000","last_became_peered":"2024-09-22T03:14:21.197364+0000","last_unstale":"2024-09-22T03:14:25.290105+0000","last_undegraded":"2024-09-22T03:14:25.290105+0000","last_fullsized":"2024-09-22T03:14:25.290105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"58'100","reported_seq":113,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:26.065428+0000","last_change":"2024-09-22T03:14:21.194514+0000","last_active":"2024-09-22T03:14:26.065428+0000","last_peered":"2024-09-22T03:14:26.065428+0000","last_clean":"2024-09-22T03:14:26.065428+0000","last_became_active":"2024-09-22T03:14:21.184663+0000","last_became_peered":"2024-09-22T03:14:21.184663+0000","last_unstale":"2024-09-22T03:14:26.065428+0000","last_undegraded":"2024-09-22T03:14:26.065428+0000","last_fullsized":"2024-09-22T03:14:26.065428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.243016+0000","last_change":"2024-09-22T03:14:21.192411+0000","last_active":"2024-09-22T03:14:22.243016+0000","last_peered":"2024-09-22T03:14:22.243016+0000","last_clean":"2024-09-22T03:14:22.243016+0000","last_became_active":"2024-09-22T03:14:21.188971+0000","last_became_peered":"2024-09-22T03:14:21.188971+0000","last_unstale":"2024-09-22T03:14:22.243016+0000","last_undegraded":"2024-09-22T03:14:22.243016+0000","last_fullsized":"2024-09-22T03:14:22.243016+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"58'67","reported_seq":80,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:25.241622+0000","last_change":"2024-09-22T03:14:21.197231+0000","last_active":"2024-09-22T03:14:25.241622+0000","last_peered":"2024-09-22T03:14:25.241622+0000","last_clean":"2024-09-22T03:14:25.241622+0000","last_became_active":"2024-09-22T03:14:21.189148+0000","last_became_peered":"2024-09-22T03:14:21.189148+0000","last_unstale":"2024-09-22T03:14:25.241622+0000","last_undegraded":"2024-09-22T03:14:25.241622+0000","last_fullsized":"2024-09-22T03:14:25.241622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"58'87","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:25.186689+0000","last_change":"2024-09-22T03:14:21.221221+0000","last_active":"2024-09-22T03:14:25.186689+0000","last_peered":"2024-09-22T03:14:25.186689+0000","last_clean":"2024-09-22T03:14:25.186689+0000","last_became_active":"2024-09-22T03:14:21.215886+0000","last_became_peered":"2024-09-22T03:14:21.215886+0000","last_unstale":"2024-09-22T03:14:25.186689+0000","last_undegraded":"2024-09-22T03:14:25.186689+0000","last_fullsized":"2024-09-22T03:14:25.186689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"58'70","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:25.303362+0000","last_change":"2024-09-22T03:14:21.221882+0000","last_active":"2024-09-22T03:14:25.303362+0000","last_peered":"2024-09-22T03:14:25.303362+0000","last_clean":"2024-09-22T03:14:25.303362+0000","last_became_active":"2024-09-22T03:14:21.217925+0000","last_became_peered":"2024-09-22T03:14:21.217925+0000","last_unstale":"2024-09-22T03:14:25.303362+0000","last_undegraded":"2024-09-22T03:14:25.303362+0000","last_fullsized":"2024-09-22T03:14:25.303362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"58'85","reported_seq":98,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:25.124710+0000","last_change":"2024-09-22T03:14:21.192905+0000","last_active":"2024-09-22T03:14:25.124710+0000","last_peered":"2024-09-22T03:14:25.124710+0000","last_clean":"2024-09-22T03:14:25.124710+0000","last_became_active":"2024-09-22T03:14:21.185129+0000","last_became_peered":"2024-09-22T03:14:21.185129+0000","last_unstale":"2024-09-22T03:14:25.124710+0000","last_undegraded":"2024-09-22T03:14:25.124710+0000","last_fullsized":"2024-09-22T03:14:25.124710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.128985+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:14:22.128985+0000","last_peered":"2024-09-22T03:14:22.128985+0000","last_clean":"2024-09-22T03:14:22.128985+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:14:22.128985+0000","last_undegraded":"2024-09-22T03:14:22.128985+0000","last_fullsized":"2024-09-22T03:14:22.128985+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"58'25","reported_seq":38,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:23.213272+0000","last_change":"2024-09-22T03:14:21.206700+0000","last_active":"2024-09-22T03:14:23.213272+0000","last_peered":"2024-09-22T03:14:23.213272+0000","last_clean":"2024-09-22T03:14:23.213272+0000","last_became_active":"2024-09-22T03:14:21.202903+0000","last_became_peered":"2024-09-22T03:14:21.202903+0000","last_unstale":"2024-09-22T03:14:23.213272+0000","last_undegraded":"2024-09-22T03:14:23.213272+0000","last_fullsized":"2024-09-22T03:14:23.213272+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":155,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.123583+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:14:22.123583+0000","last_peered":"2024-09-22T03:14:22.123583+0000","last_clean":"2024-09-22T03:14:22.123583+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:14:22.123583+0000","last_undegraded":"2024-09-22T03:14:22.123583+0000","last_fullsized":"2024-09-22T03:14:22.123583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"58'23","reported_seq":36,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:23.067821+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:14:23.067821+0000","last_peered":"2024-09-22T03:14:23.067821+0000","last_clean":"2024-09-22T03:14:23.067821+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:14:23.067821+0000","last_undegraded":"2024-09-22T03:14:23.067821+0000","last_fullsized":"2024-09-22T03:14:23.067821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.126298+0000","last_change":"2024-09-22T03:14:19.140598+0000","last_active":"2024-09-22T03:14:22.126298+0000","last_peered":"2024-09-22T03:14:22.126298+0000","last_clean":"2024-09-22T03:14:22.126298+0000","last_became_active":"2024-09-22T03:14:07.191763+0000","last_became_peered":"2024-09-22T03:14:07.191763+0000","last_unstale":"2024-09-22T03:14:22.126298+0000","last_undegraded":"2024-09-22T03:14:22.126298+0000","last_fullsized":"2024-09-22T03:14:22.126298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"58'1","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.208412+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:14:22.208412+0000","last_peered":"2024-09-22T03:14:22.208412+0000","last_clean":"2024-09-22T03:14:22.208412+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:14:22.208412+0000","last_undegraded":"2024-09-22T03:14:22.208412+0000","last_fullsized":"2024-09-22T03:14:22.208412+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129978+0000","last_change":"2024-09-22T03:14:19.157303+0000","last_active":"2024-09-22T03:14:22.129978+0000","last_peered":"2024-09-22T03:14:22.129978+0000","last_clean":"2024-09-22T03:14:22.129978+0000","last_became_active":"2024-09-22T03:14:07.230230+0000","last_became_peered":"2024-09-22T03:14:07.230230+0000","last_unstale":"2024-09-22T03:14:22.129978+0000","last_undegraded":"2024-09-22T03:14:22.129978+0000","last_fullsized":"2024-09-22T03:14:22.129978+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":279,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129224+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:14:22.129224+0000","last_peered":"2024-09-22T03:14:22.129224+0000","last_clean":"2024-09-22T03:14:22.129224+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:14:22.129224+0000","last_undegraded":"2024-09-22T03:14:22.129224+0000","last_fullsized":"2024-09-22T03:14:22.129224+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"58'74","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:25.314774+0000","last_change":"2024-09-22T03:14:21.179859+0000","last_active":"2024-09-22T03:14:25.314774+0000","last_peered":"2024-09-22T03:14:25.314774+0000","last_clean":"2024-09-22T03:14:25.314774+0000","last_became_active":"2024-09-22T03:14:21.175053+0000","last_became_peered":"2024-09-22T03:14:21.175053+0000","last_unstale":"2024-09-22T03:14:25.314774+0000","last_undegraded":"2024-09-22T03:14:25.314774+0000","last_fullsized":"2024-09-22T03:14:25.314774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.127690+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:14:22.127690+0000","last_peered":"2024-09-22T03:14:22.127690+0000","last_clean":"2024-09-22T03:14:22.127690+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:14:22.127690+0000","last_undegraded":"2024-09-22T03:14:22.127690+0000","last_fullsized":"2024-09-22T03:14:22.127690+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129748+0000","last_change":"2024-09-22T03:14:21.200985+0000","last_active":"2024-09-22T03:14:22.129748+0000","last_peered":"2024-09-22T03:14:22.129748+0000","last_clean":"2024-09-22T03:14:22.129748+0000","last_became_active":"2024-09-22T03:14:21.196935+0000","last_became_peered":"2024-09-22T03:14:21.196935+0000","last_unstale":"2024-09-22T03:14:22.129748+0000","last_undegraded":"2024-09-22T03:14:22.129748+0000","last_fullsized":"2024-09-22T03:14:22.129748+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130987+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:14:22.130987+0000","last_peered":"2024-09-22T03:14:22.130987+0000","last_clean":"2024-09-22T03:14:22.130987+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:14:22.130987+0000","last_undegraded":"2024-09-22T03:14:22.130987+0000","last_fullsized":"2024-09-22T03:14:22.130987+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.209985+0000","last_change":"2024-09-22T03:14:21.201433+0000","last_active":"2024-09-22T03:14:22.209985+0000","last_peered":"2024-09-22T03:14:22.209985+0000","last_clean":"2024-09-22T03:14:22.209985+0000","last_became_active":"2024-09-22T03:14:21.196689+0000","last_became_peered":"2024-09-22T03:14:21.196689+0000","last_unstale":"2024-09-22T03:14:22.209985+0000","last_undegraded":"2024-09-22T03:14:22.209985+0000","last_fullsized":"2024-09-22T03:14:22.209985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129348+0000","last_change":"2024-09-22T03:14:19.146918+0000","last_active":"2024-09-22T03:14:22.129348+0000","last_peered":"2024-09-22T03:14:22.129348+0000","last_clean":"2024-09-22T03:14:22.129348+0000","last_became_active":"2024-09-22T03:14:07.194059+0000","last_became_peered":"2024-09-22T03:14:07.194059+0000","last_unstale":"2024-09-22T03:14:22.129348+0000","last_undegraded":"2024-09-22T03:14:22.129348+0000","last_fullsized":"2024-09-22T03:14:22.129348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"58'96","reported_seq":109,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:25.292308+0000","last_change":"2024-09-22T03:14:21.222937+0000","last_active":"2024-09-22T03:14:25.292308+0000","last_peered":"2024-09-22T03:14:25.292308+0000","last_clean":"2024-09-22T03:14:25.292308+0000","last_became_active":"2024-09-22T03:14:21.218857+0000","last_became_peered":"2024-09-22T03:14:21.218857+0000","last_unstale":"2024-09-22T03:14:25.292308+0000","last_undegraded":"2024-09-22T03:14:25.292308+0000","last_fullsized":"2024-09-22T03:14:25.292308+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130197+0000","last_change":"2024-09-22T03:14:19.146650+0000","last_active":"2024-09-22T03:14:22.130197+0000","last_peered":"2024-09-22T03:14:22.130197+0000","last_clean":"2024-09-22T03:14:22.130197+0000","last_became_active":"2024-09-22T03:14:07.200969+0000","last_became_peered":"2024-09-22T03:14:07.200969+0000","last_unstale":"2024-09-22T03:14:22.130197+0000","last_undegraded":"2024-09-22T03:14:22.130197+0000","last_fullsized":"2024-09-22T03:14:22.130197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":51380224,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":50176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10764288,"data_stored":172228608,"data_compressed":8176161,"data_compressed_allocated":10764288,"data_compressed_original":172228608,"omap_allocated":0,"internal_metadata":0},"log_size":769,"ondisk_log_size":769,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266707,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43012,"kb_used_data":4276,"kb_used_omap":0,"kb_used_meta":38720,"kb_avail":94328828,"statfs":{"total":96636764160,"available":96592719872,"internally_reserved":0,"allocated":4378624,"data_stored":62444500,"data_compressed":2955508,"data_compressed_allocated":3891200,"data_compressed_original":62259200,"omap_allocated":0,"internal_metadata":39649280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45640,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94326200,"statfs":{"total":96636764160,"available":96590028800,"internally_reserved":0,"allocated":778240,"data_stored":4445684,"data_compressed":204123,"data_compressed_allocated":270336,"data_compressed_original":4243456,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149811,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76400,"kb_used_data":4508,"kb_used_omap":0,"kb_used_meta":71872,"kb_avail":94295440,"statfs":{"total":96636764160,"available":96558530560,"internally_reserved":0,"allocated":4616192,"data_stored":65852916,"data_compressed":3119287,"data_compressed_allocated":4108288,"data_compressed_original":65650688,"omap_allocated":0,"internal_metadata":73596928},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45404,"kb_used_data":524,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94326436,"statfs":{"total":96636764160,"available":96590270464,"internally_reserved":0,"allocated":536576,"data_stored":772044,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":1245184,"data_compressed":59084,"data_compressed_allocated":77824,"data_compressed_original":1245184,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4325376,"data_stored":69206016,"data_compressed":3285371,"data_compressed_allocated":4325376,"data_compressed_original":69206016,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":17760256,"data_compressed":843014,"data_compressed_allocated":1110016,"data_compressed_original":17760256,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5251072,"data_stored":84017152,"data_compressed":3988692,"data_compressed_allocated":5251072,"data_compressed_original":84017152,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:14:29.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 7 16 2927 2911 25.9889 24.25 0.0599006 0.0383218 2024-09-22T03:14:30.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 8 16 3342 3326 25.9822 25.9375 0.0422872 0.0384262 2024-09-22T03:14:30.595 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:30.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:30.746+0000 7fa2e0157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:30.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:30.754+0000 7fa2e0157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:31.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 9 16 3754 3738 25.9562 25.75 0.0198314 0.0384336 2024-09-22T03:14:32.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 10 16 4182 4166 26.0354 26.75 0.0358335 0.03834 2024-09-22T03:14:32.960 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:33.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:33.106+0000 7f0d07957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:33.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:33.115+0000 7f0d07957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:33.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 11 16 4592 4576 25.9979 25.625 0.0492005 0.0383977 2024-09-22T03:14:33.963 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:33.963 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:33.963 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:14:34.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:34.110+0000 7f4425c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:34.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:34.121+0000 7f4425c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:34.201 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 12 16 5001 4985 25.9614 25.5625 0.019033 0.0384451 2024-09-22T03:14:34.469 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:14:34.469 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:14:34.486 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-22T03:14:33.264490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223871539,"num_objects":3411,"num_object_clones":0,"num_object_copies":10233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3546,"num_write_kb":219944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3491,"ondisk_log_size":3491,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":269988,"kb_used_data":40484,"kb_used_omap":0,"kb_used_meta":229440,"kb_avail":377217372,"statfs":{"total":386547056640,"available":386270588928,"internally_reserved":0,"allocated":41455616,"data_stored":631723421,"data_compressed":29964898,"data_compressed_allocated":39448576,"data_compressed_original":630931456,"omap_allocated":0,"internal_metadata":234946560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":223281152,"num_objects":3407,"num_object_clones":0,"num_object_copies":10221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3407,"num_write_kb":218048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.146951"},"pg_stats":[{"pgid":"4.f","version":"58'150","reported_seq":163,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:28.266622+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:14:28.266622+0000","last_peered":"2024-09-22T03:14:28.266622+0000","last_clean":"2024-09-22T03:14:28.266622+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:14:28.266622+0000","last_undegraded":"2024-09-22T03:14:28.266622+0000","last_fullsized":"2024-09-22T03:14:28.266622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9895936,"num_objects":151,"num_object_clones":0,"num_object_copies":453,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":9664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"58'148","reported_seq":161,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:28.237709+0000","last_change":"2024-09-22T03:14:21.195156+0000","last_active":"2024-09-22T03:14:28.237709+0000","last_peered":"2024-09-22T03:14:28.237709+0000","last_clean":"2024-09-22T03:14:28.237709+0000","last_became_active":"2024-09-22T03:14:21.188092+0000","last_became_peered":"2024-09-22T03:14:21.188092+0000","last_unstale":"2024-09-22T03:14:28.237709+0000","last_undegraded":"2024-09-22T03:14:28.237709+0000","last_fullsized":"2024-09-22T03:14:28.237709+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764864,"num_objects":149,"num_object_clones":0,"num_object_copies":447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":149,"num_write_kb":9536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"58'222","reported_seq":235,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.278016+0000","last_change":"2024-09-22T03:14:21.204545+0000","last_active":"2024-09-22T03:14:30.278016+0000","last_peered":"2024-09-22T03:14:30.278016+0000","last_clean":"2024-09-22T03:14:30.278016+0000","last_became_active":"2024-09-22T03:14:21.197364+0000","last_became_peered":"2024-09-22T03:14:21.197364+0000","last_unstale":"2024-09-22T03:14:30.278016+0000","last_undegraded":"2024-09-22T03:14:30.278016+0000","last_fullsized":"2024-09-22T03:14:30.278016+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14614528,"num_objects":223,"num_object_clones":0,"num_object_copies":669,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":14272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"58'226","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.967138+0000","last_change":"2024-09-22T03:14:21.194514+0000","last_active":"2024-09-22T03:14:30.967138+0000","last_peered":"2024-09-22T03:14:30.967138+0000","last_clean":"2024-09-22T03:14:30.967138+0000","last_became_active":"2024-09-22T03:14:21.184663+0000","last_became_peered":"2024-09-22T03:14:21.184663+0000","last_unstale":"2024-09-22T03:14:30.967138+0000","last_undegraded":"2024-09-22T03:14:30.967138+0000","last_fullsized":"2024-09-22T03:14:30.967138+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14876672,"num_objects":227,"num_object_clones":0,"num_object_copies":681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":14528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"58'270","reported_seq":283,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:32.192704+0000","last_change":"2024-09-22T03:14:21.192411+0000","last_active":"2024-09-22T03:14:32.192704+0000","last_peered":"2024-09-22T03:14:32.192704+0000","last_clean":"2024-09-22T03:14:32.192704+0000","last_became_active":"2024-09-22T03:14:21.188971+0000","last_became_peered":"2024-09-22T03:14:21.188971+0000","last_unstale":"2024-09-22T03:14:32.192704+0000","last_undegraded":"2024-09-22T03:14:32.192704+0000","last_fullsized":"2024-09-22T03:14:32.192704+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":271,"num_object_clones":0,"num_object_copies":813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"58'205","reported_seq":218,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.292268+0000","last_change":"2024-09-22T03:14:21.197231+0000","last_active":"2024-09-22T03:14:30.292268+0000","last_peered":"2024-09-22T03:14:30.292268+0000","last_clean":"2024-09-22T03:14:30.292268+0000","last_became_active":"2024-09-22T03:14:21.189148+0000","last_became_peered":"2024-09-22T03:14:21.189148+0000","last_unstale":"2024-09-22T03:14:30.292268+0000","last_undegraded":"2024-09-22T03:14:30.292268+0000","last_fullsized":"2024-09-22T03:14:30.292268+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":206,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":206,"num_write_kb":13184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"58'226","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.239008+0000","last_change":"2024-09-22T03:14:21.221221+0000","last_active":"2024-09-22T03:14:30.239008+0000","last_peered":"2024-09-22T03:14:30.239008+0000","last_clean":"2024-09-22T03:14:30.239008+0000","last_became_active":"2024-09-22T03:14:21.215886+0000","last_became_peered":"2024-09-22T03:14:21.215886+0000","last_unstale":"2024-09-22T03:14:30.239008+0000","last_undegraded":"2024-09-22T03:14:30.239008+0000","last_fullsized":"2024-09-22T03:14:30.239008+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14876672,"num_objects":227,"num_object_clones":0,"num_object_copies":681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":14528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"58'200","reported_seq":213,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.180562+0000","last_change":"2024-09-22T03:14:21.221882+0000","last_active":"2024-09-22T03:14:30.180562+0000","last_peered":"2024-09-22T03:14:30.180562+0000","last_clean":"2024-09-22T03:14:30.180562+0000","last_became_active":"2024-09-22T03:14:21.217925+0000","last_became_peered":"2024-09-22T03:14:21.217925+0000","last_unstale":"2024-09-22T03:14:30.180562+0000","last_undegraded":"2024-09-22T03:14:30.180562+0000","last_fullsized":"2024-09-22T03:14:30.180562+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":201,"num_object_clones":0,"num_object_copies":603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":201,"num_write_kb":12864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"58'212","reported_seq":225,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.241470+0000","last_change":"2024-09-22T03:14:21.192905+0000","last_active":"2024-09-22T03:14:30.241470+0000","last_peered":"2024-09-22T03:14:30.241470+0000","last_clean":"2024-09-22T03:14:30.241470+0000","last_became_active":"2024-09-22T03:14:21.185129+0000","last_became_peered":"2024-09-22T03:14:21.185129+0000","last_unstale":"2024-09-22T03:14:30.241470+0000","last_undegraded":"2024-09-22T03:14:30.241470+0000","last_fullsized":"2024-09-22T03:14:30.241470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":213,"num_object_clones":0,"num_object_copies":639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":213,"num_write_kb":13632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.128985+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:14:22.128985+0000","last_peered":"2024-09-22T03:14:22.128985+0000","last_clean":"2024-09-22T03:14:22.128985+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:14:22.128985+0000","last_undegraded":"2024-09-22T03:14:22.128985+0000","last_fullsized":"2024-09-22T03:14:22.128985+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"58'148","reported_seq":161,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:28.223106+0000","last_change":"2024-09-22T03:14:21.206700+0000","last_active":"2024-09-22T03:14:28.223106+0000","last_peered":"2024-09-22T03:14:28.223106+0000","last_clean":"2024-09-22T03:14:28.223106+0000","last_became_active":"2024-09-22T03:14:21.202903+0000","last_became_peered":"2024-09-22T03:14:21.202903+0000","last_unstale":"2024-09-22T03:14:28.223106+0000","last_undegraded":"2024-09-22T03:14:28.223106+0000","last_fullsized":"2024-09-22T03:14:28.223106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764864,"num_objects":149,"num_object_clones":0,"num_object_copies":447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":149,"num_write_kb":9536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":155,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.123583+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:14:22.123583+0000","last_peered":"2024-09-22T03:14:22.123583+0000","last_clean":"2024-09-22T03:14:22.123583+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:14:22.123583+0000","last_undegraded":"2024-09-22T03:14:22.123583+0000","last_fullsized":"2024-09-22T03:14:22.123583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"58'165","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:28.226176+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:14:28.226176+0000","last_peered":"2024-09-22T03:14:28.226176+0000","last_clean":"2024-09-22T03:14:28.226176+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:14:28.226176+0000","last_undegraded":"2024-09-22T03:14:28.226176+0000","last_fullsized":"2024-09-22T03:14:28.226176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10878976,"num_objects":166,"num_object_clones":0,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":10624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.126298+0000","last_change":"2024-09-22T03:14:19.140598+0000","last_active":"2024-09-22T03:14:22.126298+0000","last_peered":"2024-09-22T03:14:22.126298+0000","last_clean":"2024-09-22T03:14:22.126298+0000","last_became_active":"2024-09-22T03:14:07.191763+0000","last_became_peered":"2024-09-22T03:14:07.191763+0000","last_unstale":"2024-09-22T03:14:22.126298+0000","last_undegraded":"2024-09-22T03:14:22.126298+0000","last_fullsized":"2024-09-22T03:14:22.126298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"58'269","reported_seq":282,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:32.233764+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:14:32.233764+0000","last_peered":"2024-09-22T03:14:32.233764+0000","last_clean":"2024-09-22T03:14:32.233764+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:14:32.233764+0000","last_undegraded":"2024-09-22T03:14:32.233764+0000","last_fullsized":"2024-09-22T03:14:32.233764+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129978+0000","last_change":"2024-09-22T03:14:19.157303+0000","last_active":"2024-09-22T03:14:22.129978+0000","last_peered":"2024-09-22T03:14:22.129978+0000","last_clean":"2024-09-22T03:14:22.129978+0000","last_became_active":"2024-09-22T03:14:07.230230+0000","last_became_peered":"2024-09-22T03:14:07.230230+0000","last_unstale":"2024-09-22T03:14:22.129978+0000","last_undegraded":"2024-09-22T03:14:22.129978+0000","last_fullsized":"2024-09-22T03:14:22.129978+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":279,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129224+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:14:22.129224+0000","last_peered":"2024-09-22T03:14:22.129224+0000","last_clean":"2024-09-22T03:14:22.129224+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:14:22.129224+0000","last_undegraded":"2024-09-22T03:14:22.129224+0000","last_fullsized":"2024-09-22T03:14:22.129224+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"58'213","reported_seq":226,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.200334+0000","last_change":"2024-09-22T03:14:21.179859+0000","last_active":"2024-09-22T03:14:30.200334+0000","last_peered":"2024-09-22T03:14:30.200334+0000","last_clean":"2024-09-22T03:14:30.200334+0000","last_became_active":"2024-09-22T03:14:21.175053+0000","last_became_peered":"2024-09-22T03:14:21.175053+0000","last_unstale":"2024-09-22T03:14:30.200334+0000","last_undegraded":"2024-09-22T03:14:30.200334+0000","last_fullsized":"2024-09-22T03:14:30.200334+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14024704,"num_objects":214,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":214,"num_write_kb":13696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.127690+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:14:22.127690+0000","last_peered":"2024-09-22T03:14:22.127690+0000","last_clean":"2024-09-22T03:14:22.127690+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:14:22.127690+0000","last_undegraded":"2024-09-22T03:14:22.127690+0000","last_fullsized":"2024-09-22T03:14:22.127690+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"58'235","reported_seq":248,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:32.182990+0000","last_change":"2024-09-22T03:14:21.200985+0000","last_active":"2024-09-22T03:14:32.182990+0000","last_peered":"2024-09-22T03:14:32.182990+0000","last_clean":"2024-09-22T03:14:32.182990+0000","last_became_active":"2024-09-22T03:14:21.196935+0000","last_became_peered":"2024-09-22T03:14:21.196935+0000","last_unstale":"2024-09-22T03:14:32.182990+0000","last_undegraded":"2024-09-22T03:14:32.182990+0000","last_fullsized":"2024-09-22T03:14:32.182990+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15466496,"num_objects":236,"num_object_clones":0,"num_object_copies":708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":15104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130987+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:14:22.130987+0000","last_peered":"2024-09-22T03:14:22.130987+0000","last_clean":"2024-09-22T03:14:22.130987+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:14:22.130987+0000","last_undegraded":"2024-09-22T03:14:22.130987+0000","last_fullsized":"2024-09-22T03:14:22.130987+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"58'273","reported_seq":286,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:32.210762+0000","last_change":"2024-09-22T03:14:21.201433+0000","last_active":"2024-09-22T03:14:32.210762+0000","last_peered":"2024-09-22T03:14:32.210762+0000","last_clean":"2024-09-22T03:14:32.210762+0000","last_became_active":"2024-09-22T03:14:21.196689+0000","last_became_peered":"2024-09-22T03:14:21.196689+0000","last_unstale":"2024-09-22T03:14:32.210762+0000","last_undegraded":"2024-09-22T03:14:32.210762+0000","last_fullsized":"2024-09-22T03:14:32.210762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17956864,"num_objects":274,"num_object_clones":0,"num_object_copies":822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":274,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129348+0000","last_change":"2024-09-22T03:14:19.146918+0000","last_active":"2024-09-22T03:14:22.129348+0000","last_peered":"2024-09-22T03:14:22.129348+0000","last_clean":"2024-09-22T03:14:22.129348+0000","last_became_active":"2024-09-22T03:14:07.194059+0000","last_became_peered":"2024-09-22T03:14:07.194059+0000","last_unstale":"2024-09-22T03:14:22.129348+0000","last_undegraded":"2024-09-22T03:14:22.129348+0000","last_fullsized":"2024-09-22T03:14:22.129348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"58'229","reported_seq":242,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:30.287700+0000","last_change":"2024-09-22T03:14:21.222937+0000","last_active":"2024-09-22T03:14:30.287700+0000","last_peered":"2024-09-22T03:14:30.287700+0000","last_clean":"2024-09-22T03:14:30.287700+0000","last_became_active":"2024-09-22T03:14:21.218857+0000","last_became_peered":"2024-09-22T03:14:21.218857+0000","last_unstale":"2024-09-22T03:14:30.287700+0000","last_undegraded":"2024-09-22T03:14:30.287700+0000","last_fullsized":"2024-09-22T03:14:30.287700+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":230,"num_write_kb":14720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130197+0000","last_change":"2024-09-22T03:14:19.146650+0000","last_active":"2024-09-22T03:14:22.130197+0000","last_peered":"2024-09-22T03:14:22.130197+0000","last_clean":"2024-09-22T03:14:22.130197+0000","last_became_active":"2024-09-22T03:14:07.200969+0000","last_became_peered":"2024-09-22T03:14:07.200969+0000","last_unstale":"2024-09-22T03:14:22.130197+0000","last_undegraded":"2024-09-22T03:14:22.130197+0000","last_fullsized":"2024-09-22T03:14:22.130197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":223281152,"num_objects":3407,"num_object_clones":0,"num_object_copies":10221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3407,"num_write_kb":218048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42987520,"data_stored":687800320,"data_compressed":32657110,"data_compressed_allocated":42987520,"data_compressed_original":687800320,"omap_allocated":0,"internal_metadata":0},"log_size":3391,"ondisk_log_size":3391,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266712,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58108,"kb_used_data":11052,"kb_used_omap":0,"kb_used_meta":47040,"kb_avail":94313732,"statfs":{"total":96636764160,"available":96577261568,"internally_reserved":0,"allocated":11317248,"data_stored":173462484,"data_compressed":8227236,"data_compressed_allocated":10829824,"data_compressed_original":173277184,"omap_allocated":0,"internal_metadata":48168960},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59308,"kb_used_data":6108,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312532,"statfs":{"total":96636764160,"available":96576032768,"internally_reserved":0,"allocated":6254592,"data_stored":92067316,"data_compressed":4364287,"data_compressed_allocated":5746688,"data_compressed_original":91865088,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149816,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87392,"kb_used_data":11344,"kb_used_omap":0,"kb_used_meta":76032,"kb_avail":94284448,"statfs":{"total":96636764160,"available":96547274752,"internally_reserved":0,"allocated":11616256,"data_stored":177853940,"data_compressed":8437695,"data_compressed_allocated":11108352,"data_compressed_original":177651712,"omap_allocated":0,"internal_metadata":77856768},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65180,"kb_used_data":11980,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94306660,"statfs":{"total":96636764160,"available":96570019840,"internally_reserved":0,"allocated":12267520,"data_stored":188339681,"data_compressed":8935680,"data_compressed_allocated":11763712,"data_compressed_original":188137472,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12840960,"data_stored":205455360,"data_compressed":9755284,"data_compressed_allocated":12840960,"data_compressed_original":205455360,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11292672,"data_stored":180682752,"data_compressed":8578883,"data_compressed_allocated":11292672,"data_compressed_original":180682752,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6639616,"data_stored":106233856,"data_compressed":5043851,"data_compressed_allocated":6639616,"data_compressed_original":106233856,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12214272,"data_stored":195428352,"data_compressed":9279092,"data_compressed_allocated":12214272,"data_compressed_original":195428352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:14:35.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 13 16 5401 5385 25.8873 25 0.0346503 0.0385655 2024-09-22T03:14:35.309 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:35.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:35.459+0000 7f4e71c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:35.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:35.469+0000 7f4e71c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:36.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 14 16 5803 5787 25.8327 25.125 0.0143729 0.0386782 2024-09-22T03:14:37.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 15 16 6204 6188 25.7812 25.0625 0.0225966 0.0386838 2024-09-22T03:14:37.708 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:37.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:37.865+0000 7f2d6fb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:37.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:37.874+0000 7f2d6fb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:38.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 16 16 6608 6592 25.7479 25.25 0.0329826 0.0387875 2024-09-22T03:14:39.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 17 16 7002 6986 25.6817 24.625 0.0287537 0.0388837 2024-09-22T03:14:39.488 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:39.488 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:39.488 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:14:39.488 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:14:39.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:39.648+0000 7fadc68d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:39.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:39.661+0000 7fadc68d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:39.697 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:39.702 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:39.702 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:39.703 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:14:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:39.851+0000 7f2aec557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:39.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:39.865+0000 7f2aec557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:39.940 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:39.940 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:39.940 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:39.941 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:39.945 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:39.946 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:39.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:14:40.073 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:40.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:40.109+0000 7fbc69f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:40.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:40.124+0000 7fbc69f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:40.192 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:40.192 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:40.192 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:40.192 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:40.192 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:40.192 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:40.193 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:40.193 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:40.193 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:40.193 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:40.193 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:40.193 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:40.196 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:40.197 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:40.197 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:14:40.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 18 16 7369 7353 25.5291 22.9375 0.0774322 0.0391006 2024-09-22T03:14:40.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:40.232+0000 7fa4e3f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:40.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:40.242+0000 7fa4e3f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:40.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:40.366+0000 7fa72ae2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:40.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:40.384+0000 7fa72ae2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:40.420 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:14:40.420 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:14:40.421 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:14:40.425 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:14:40.426 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:14:41.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 19 16 7741 7725 25.4091 23.25 0.0371145 0.0392987 2024-09-22T03:14:42.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:14:42.201875+0000 min lat: 0.00928913 max lat: 0.1236 avg lat: 0.0393284 2024-09-22T03:14:42.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:14:42.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 20 16 8142 8126 25.3917 25.0625 0.036668 0.0393284 2024-09-22T03:14:42.417 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:42.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:42.558+0000 7f52ece18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:42.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:42.569+0000 7f52ece18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:43.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 21 16 8521 8505 25.3104 23.6875 0.0561681 0.0394721 2024-09-22T03:14:44.202 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 22 16 8909 8893 25.2621 24.25 0.0536334 0.0395288 2024-09-22T03:14:44.845 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:44.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:44.994+0000 7f1cfad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:45.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:45.001+0000 7f1cfad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:45.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 23 16 9265 9249 25.1311 22.25 0.0515897 0.0397415 2024-09-22T03:14:45.426 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:45.427 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:45.427 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:14:45.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:45.568+0000 7f5ade397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:45.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:45.581+0000 7f5ade397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:45.887 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:14:45.887 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:14:45.906 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-22T03:14:45.266819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":518521395,"num_objects":7907,"num_object_clones":0,"num_object_copies":23721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8042,"num_write_kb":507688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7987,"ondisk_log_size":7987,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":324812,"kb_used_data":93900,"kb_used_omap":0,"kb_used_meta":230848,"kb_avail":377162548,"statfs":{"total":386547056640,"available":386214449152,"internally_reserved":0,"allocated":96153600,"data_stored":1506891165,"data_compressed":71522546,"data_compressed_allocated":94146560,"data_compressed_original":1506099200,"omap_allocated":0,"internal_metadata":236388352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":294649856,"num_objects":4496,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":287744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002329"},"pg_stats":[{"pgid":"4.f","version":"58'515","reported_seq":528,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:43.219614+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:14:43.219614+0000","last_peered":"2024-09-22T03:14:43.219614+0000","last_clean":"2024-09-22T03:14:43.219614+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:14:43.219614+0000","last_undegraded":"2024-09-22T03:14:43.219614+0000","last_fullsized":"2024-09-22T03:14:43.219614+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33816576,"num_objects":516,"num_object_clones":0,"num_object_copies":1548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":516,"num_write_kb":33024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"58'552","reported_seq":565,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:43.124281+0000","last_change":"2024-09-22T03:14:21.195156+0000","last_active":"2024-09-22T03:14:43.124281+0000","last_peered":"2024-09-22T03:14:43.124281+0000","last_clean":"2024-09-22T03:14:43.124281+0000","last_became_active":"2024-09-22T03:14:21.188092+0000","last_became_peered":"2024-09-22T03:14:21.188092+0000","last_unstale":"2024-09-22T03:14:43.124281+0000","last_undegraded":"2024-09-22T03:14:43.124281+0000","last_fullsized":"2024-09-22T03:14:43.124281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36241408,"num_objects":553,"num_object_clones":0,"num_object_copies":1659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":35392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"58'472","reported_seq":485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.290409+0000","last_change":"2024-09-22T03:14:21.204545+0000","last_active":"2024-09-22T03:14:40.290409+0000","last_peered":"2024-09-22T03:14:40.290409+0000","last_clean":"2024-09-22T03:14:40.290409+0000","last_became_active":"2024-09-22T03:14:21.197364+0000","last_became_peered":"2024-09-22T03:14:21.197364+0000","last_unstale":"2024-09-22T03:14:40.290409+0000","last_undegraded":"2024-09-22T03:14:40.290409+0000","last_fullsized":"2024-09-22T03:14:40.290409+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":473,"num_object_clones":0,"num_object_copies":1419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"58'454","reported_seq":467,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.979499+0000","last_change":"2024-09-22T03:14:21.194514+0000","last_active":"2024-09-22T03:14:40.979499+0000","last_peered":"2024-09-22T03:14:40.979499+0000","last_clean":"2024-09-22T03:14:40.979499+0000","last_became_active":"2024-09-22T03:14:21.184663+0000","last_became_peered":"2024-09-22T03:14:21.184663+0000","last_unstale":"2024-09-22T03:14:40.979499+0000","last_undegraded":"2024-09-22T03:14:40.979499+0000","last_fullsized":"2024-09-22T03:14:40.979499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29818880,"num_objects":455,"num_object_clones":0,"num_object_copies":1365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":455,"num_write_kb":29120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"58'520","reported_seq":533,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:42.188288+0000","last_change":"2024-09-22T03:14:21.192411+0000","last_active":"2024-09-22T03:14:42.188288+0000","last_peered":"2024-09-22T03:14:42.188288+0000","last_clean":"2024-09-22T03:14:42.188288+0000","last_became_active":"2024-09-22T03:14:21.188971+0000","last_became_peered":"2024-09-22T03:14:21.188971+0000","last_unstale":"2024-09-22T03:14:42.188288+0000","last_undegraded":"2024-09-22T03:14:42.188288+0000","last_fullsized":"2024-09-22T03:14:42.188288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":521,"num_object_clones":0,"num_object_copies":1563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":521,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"58'466","reported_seq":479,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.247184+0000","last_change":"2024-09-22T03:14:21.197231+0000","last_active":"2024-09-22T03:14:40.247184+0000","last_peered":"2024-09-22T03:14:40.247184+0000","last_clean":"2024-09-22T03:14:40.247184+0000","last_became_active":"2024-09-22T03:14:21.189148+0000","last_became_peered":"2024-09-22T03:14:21.189148+0000","last_unstale":"2024-09-22T03:14:40.247184+0000","last_undegraded":"2024-09-22T03:14:40.247184+0000","last_fullsized":"2024-09-22T03:14:40.247184+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":467,"num_object_clones":0,"num_object_copies":1401,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":29888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"58'490","reported_seq":503,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.213668+0000","last_change":"2024-09-22T03:14:21.221221+0000","last_active":"2024-09-22T03:14:40.213668+0000","last_peered":"2024-09-22T03:14:40.213668+0000","last_clean":"2024-09-22T03:14:40.213668+0000","last_became_active":"2024-09-22T03:14:21.215886+0000","last_became_peered":"2024-09-22T03:14:21.215886+0000","last_unstale":"2024-09-22T03:14:40.213668+0000","last_undegraded":"2024-09-22T03:14:40.213668+0000","last_fullsized":"2024-09-22T03:14:40.213668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32178176,"num_objects":491,"num_object_clones":0,"num_object_copies":1473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":491,"num_write_kb":31424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"58'447","reported_seq":460,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.279555+0000","last_change":"2024-09-22T03:14:21.221882+0000","last_active":"2024-09-22T03:14:40.279555+0000","last_peered":"2024-09-22T03:14:40.279555+0000","last_clean":"2024-09-22T03:14:40.279555+0000","last_became_active":"2024-09-22T03:14:21.217925+0000","last_became_peered":"2024-09-22T03:14:21.217925+0000","last_unstale":"2024-09-22T03:14:40.279555+0000","last_undegraded":"2024-09-22T03:14:40.279555+0000","last_fullsized":"2024-09-22T03:14:40.279555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":448,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"58'442","reported_seq":455,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.173923+0000","last_change":"2024-09-22T03:14:21.192905+0000","last_active":"2024-09-22T03:14:40.173923+0000","last_peered":"2024-09-22T03:14:40.173923+0000","last_clean":"2024-09-22T03:14:40.173923+0000","last_became_active":"2024-09-22T03:14:21.185129+0000","last_became_peered":"2024-09-22T03:14:21.185129+0000","last_unstale":"2024-09-22T03:14:40.173923+0000","last_undegraded":"2024-09-22T03:14:40.173923+0000","last_fullsized":"2024-09-22T03:14:40.173923+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032448,"num_objects":443,"num_object_clones":0,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":443,"num_write_kb":28352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.128985+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:14:22.128985+0000","last_peered":"2024-09-22T03:14:22.128985+0000","last_clean":"2024-09-22T03:14:22.128985+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:14:22.128985+0000","last_undegraded":"2024-09-22T03:14:22.128985+0000","last_fullsized":"2024-09-22T03:14:22.128985+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"58'539","reported_seq":552,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:43.188587+0000","last_change":"2024-09-22T03:14:21.206700+0000","last_active":"2024-09-22T03:14:43.188587+0000","last_peered":"2024-09-22T03:14:43.188587+0000","last_clean":"2024-09-22T03:14:43.188587+0000","last_became_active":"2024-09-22T03:14:21.202903+0000","last_became_peered":"2024-09-22T03:14:21.202903+0000","last_unstale":"2024-09-22T03:14:43.188587+0000","last_undegraded":"2024-09-22T03:14:43.188587+0000","last_fullsized":"2024-09-22T03:14:43.188587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":540,"num_write_kb":34560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":155,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.123583+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:14:22.123583+0000","last_peered":"2024-09-22T03:14:22.123583+0000","last_clean":"2024-09-22T03:14:22.123583+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:14:22.123583+0000","last_undegraded":"2024-09-22T03:14:22.123583+0000","last_fullsized":"2024-09-22T03:14:22.123583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"58'518","reported_seq":531,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:43.131697+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:14:43.131697+0000","last_peered":"2024-09-22T03:14:43.131697+0000","last_clean":"2024-09-22T03:14:43.131697+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:14:43.131697+0000","last_undegraded":"2024-09-22T03:14:43.131697+0000","last_fullsized":"2024-09-22T03:14:43.131697+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34013184,"num_objects":519,"num_object_clones":0,"num_object_copies":1557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":519,"num_write_kb":33216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.126298+0000","last_change":"2024-09-22T03:14:19.140598+0000","last_active":"2024-09-22T03:14:22.126298+0000","last_peered":"2024-09-22T03:14:22.126298+0000","last_clean":"2024-09-22T03:14:22.126298+0000","last_became_active":"2024-09-22T03:14:07.191763+0000","last_became_peered":"2024-09-22T03:14:07.191763+0000","last_unstale":"2024-09-22T03:14:22.126298+0000","last_undegraded":"2024-09-22T03:14:22.126298+0000","last_fullsized":"2024-09-22T03:14:22.126298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"58'524","reported_seq":537,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:42.242908+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:14:42.242908+0000","last_peered":"2024-09-22T03:14:42.242908+0000","last_clean":"2024-09-22T03:14:42.242908+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:14:42.242908+0000","last_undegraded":"2024-09-22T03:14:42.242908+0000","last_fullsized":"2024-09-22T03:14:42.242908+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34406400,"num_objects":525,"num_object_clones":0,"num_object_copies":1575,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":525,"num_write_kb":33600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129978+0000","last_change":"2024-09-22T03:14:19.157303+0000","last_active":"2024-09-22T03:14:22.129978+0000","last_peered":"2024-09-22T03:14:22.129978+0000","last_clean":"2024-09-22T03:14:22.129978+0000","last_became_active":"2024-09-22T03:14:07.230230+0000","last_became_peered":"2024-09-22T03:14:07.230230+0000","last_unstale":"2024-09-22T03:14:22.129978+0000","last_undegraded":"2024-09-22T03:14:22.129978+0000","last_fullsized":"2024-09-22T03:14:22.129978+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":279,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129224+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:14:22.129224+0000","last_peered":"2024-09-22T03:14:22.129224+0000","last_clean":"2024-09-22T03:14:22.129224+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:14:22.129224+0000","last_undegraded":"2024-09-22T03:14:22.129224+0000","last_fullsized":"2024-09-22T03:14:22.129224+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"58'457","reported_seq":470,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.221858+0000","last_change":"2024-09-22T03:14:21.179859+0000","last_active":"2024-09-22T03:14:40.221858+0000","last_peered":"2024-09-22T03:14:40.221858+0000","last_clean":"2024-09-22T03:14:40.221858+0000","last_became_active":"2024-09-22T03:14:21.175053+0000","last_became_peered":"2024-09-22T03:14:21.175053+0000","last_unstale":"2024-09-22T03:14:40.221858+0000","last_undegraded":"2024-09-22T03:14:40.221858+0000","last_fullsized":"2024-09-22T03:14:40.221858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30015488,"num_objects":458,"num_object_clones":0,"num_object_copies":1374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":458,"num_write_kb":29312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.127690+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:14:22.127690+0000","last_peered":"2024-09-22T03:14:22.127690+0000","last_clean":"2024-09-22T03:14:22.127690+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:14:22.127690+0000","last_undegraded":"2024-09-22T03:14:22.127690+0000","last_fullsized":"2024-09-22T03:14:22.127690+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"58'480","reported_seq":493,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:42.127797+0000","last_change":"2024-09-22T03:14:21.200985+0000","last_active":"2024-09-22T03:14:42.127797+0000","last_peered":"2024-09-22T03:14:42.127797+0000","last_clean":"2024-09-22T03:14:42.127797+0000","last_became_active":"2024-09-22T03:14:21.196935+0000","last_became_peered":"2024-09-22T03:14:21.196935+0000","last_unstale":"2024-09-22T03:14:42.127797+0000","last_undegraded":"2024-09-22T03:14:42.127797+0000","last_fullsized":"2024-09-22T03:14:42.127797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31522816,"num_objects":481,"num_object_clones":0,"num_object_copies":1443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":481,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130987+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:14:22.130987+0000","last_peered":"2024-09-22T03:14:22.130987+0000","last_clean":"2024-09-22T03:14:22.130987+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:14:22.130987+0000","last_undegraded":"2024-09-22T03:14:22.130987+0000","last_fullsized":"2024-09-22T03:14:22.130987+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"58'527","reported_seq":540,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:42.124808+0000","last_change":"2024-09-22T03:14:21.201433+0000","last_active":"2024-09-22T03:14:42.124808+0000","last_peered":"2024-09-22T03:14:42.124808+0000","last_clean":"2024-09-22T03:14:42.124808+0000","last_became_active":"2024-09-22T03:14:21.196689+0000","last_became_peered":"2024-09-22T03:14:21.196689+0000","last_unstale":"2024-09-22T03:14:42.124808+0000","last_undegraded":"2024-09-22T03:14:42.124808+0000","last_fullsized":"2024-09-22T03:14:42.124808+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":528,"num_object_clones":0,"num_object_copies":1584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129348+0000","last_change":"2024-09-22T03:14:19.146918+0000","last_active":"2024-09-22T03:14:22.129348+0000","last_peered":"2024-09-22T03:14:22.129348+0000","last_clean":"2024-09-22T03:14:22.129348+0000","last_became_active":"2024-09-22T03:14:07.194059+0000","last_became_peered":"2024-09-22T03:14:07.194059+0000","last_unstale":"2024-09-22T03:14:22.129348+0000","last_undegraded":"2024-09-22T03:14:22.129348+0000","last_fullsized":"2024-09-22T03:14:22.129348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"58'484","reported_seq":497,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:40.280322+0000","last_change":"2024-09-22T03:14:21.222937+0000","last_active":"2024-09-22T03:14:40.280322+0000","last_peered":"2024-09-22T03:14:40.280322+0000","last_clean":"2024-09-22T03:14:40.280322+0000","last_became_active":"2024-09-22T03:14:21.218857+0000","last_became_peered":"2024-09-22T03:14:21.218857+0000","last_unstale":"2024-09-22T03:14:40.280322+0000","last_undegraded":"2024-09-22T03:14:40.280322+0000","last_fullsized":"2024-09-22T03:14:40.280322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":485,"num_object_clones":0,"num_object_copies":1455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":485,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130197+0000","last_change":"2024-09-22T03:14:19.146650+0000","last_active":"2024-09-22T03:14:22.130197+0000","last_peered":"2024-09-22T03:14:22.130197+0000","last_clean":"2024-09-22T03:14:22.130197+0000","last_became_active":"2024-09-22T03:14:07.200969+0000","last_became_peered":"2024-09-22T03:14:07.200969+0000","last_unstale":"2024-09-22T03:14:22.130197+0000","last_undegraded":"2024-09-22T03:14:22.130197+0000","last_fullsized":"2024-09-22T03:14:22.130197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":517931008,"num_objects":7903,"num_object_clones":0,"num_object_copies":23709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97218560,"data_stored":1555496960,"data_compressed":73859990,"data_compressed_allocated":97218560,"data_compressed_original":1555496960,"omap_allocated":0,"internal_metadata":0},"log_size":7887,"ondisk_log_size":7887,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266722,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67056,"kb_used_data":24160,"kb_used_omap":0,"kb_used_meta":42880,"kb_avail":94304784,"statfs":{"total":96636764160,"available":96568098816,"internally_reserved":0,"allocated":24739840,"data_stored":388223956,"data_compressed":18425260,"data_compressed_allocated":24252416,"data_compressed_original":388038656,"omap_allocated":0,"internal_metadata":43909120},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673179,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70484,"kb_used_data":21444,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94301356,"statfs":{"total":96636764160,"available":96564588544,"internally_reserved":0,"allocated":21958656,"data_stored":343332340,"data_compressed":16295695,"data_compressed_allocated":21450752,"data_compressed_original":343130112,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149826,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96356,"kb_used_data":24468,"kb_used_omap":0,"kb_used_meta":71872,"kb_avail":94275484,"statfs":{"total":96636764160,"available":96538095616,"internally_reserved":0,"allocated":25055232,"data_stored":392877556,"data_compressed":18648167,"data_compressed_allocated":24547328,"data_compressed_original":392675328,"omap_allocated":0,"internal_metadata":73596928},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90916,"kb_used_data":23828,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94280924,"statfs":{"total":96636764160,"available":96543666176,"internally_reserved":0,"allocated":24399872,"data_stored":382457313,"data_compressed":18153424,"data_compressed_allocated":23896064,"data_compressed_original":382255104,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24870912,"data_stored":397934592,"data_compressed":18895228,"data_compressed_allocated":24870912,"data_compressed_original":397934592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24682496,"data_stored":394919936,"data_compressed":18752011,"data_compressed_allocated":24682496,"data_compressed_original":394919936,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22249472,"data_stored":355991552,"data_compressed":16903683,"data_compressed_allocated":22249472,"data_compressed_original":355991552,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25415680,"data_stored":406650880,"data_compressed":19309068,"data_compressed_allocated":25415680,"data_compressed_original":406650880,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:14:46.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 24 16 9622 9606 25.0135 22.3125 0.0321664 0.0399326 2024-09-22T03:14:47.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 25 16 10004 9988 24.9679 23.875 0.0436219 0.0400075 2024-09-22T03:14:47.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:47.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:47.402+0000 7fb854d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:47.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:47.412+0000 7fb854d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:48.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 26 16 10364 10348 24.8729 22.5 0.0239545 0.0401566 2024-09-22T03:14:49.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 27 16 10737 10721 24.8151 23.3125 0.0559938 0.0402574 2024-09-22T03:14:49.654 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:49.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:49.800+0000 7fc7cc397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:49.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:49.807+0000 7fc7cc397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:50.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 28 16 11107 11091 24.7546 23.125 0.0299417 0.0403638 2024-09-22T03:14:50.909 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:50.909 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:50.909 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:14:51.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:51.071+0000 7fe9dbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:51.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:51.084+0000 7fe9dbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:51.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 29 16 11476 11460 24.6962 23.0625 0.0403289 0.0404657 2024-09-22T03:14:51.430 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:14:51.431 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:14:51.451 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-22T03:14:51.268019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":701760051,"num_objects":10703,"num_object_clones":0,"num_object_copies":32109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10838,"num_write_kb":686632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10783,"ondisk_log_size":10783,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":373148,"kb_used_data":128336,"kb_used_omap":0,"kb_used_meta":244736,"kb_avail":377114212,"statfs":{"total":386547056640,"available":386164953088,"internally_reserved":0,"allocated":131416064,"data_stored":2071090589,"data_compressed":98313754,"data_compressed_allocated":129409024,"data_compressed_original":2070298624,"omap_allocated":0,"internal_metadata":250609664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":310837248,"num_objects":4743,"num_object_clones":0,"num_object_copies":14229,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4743,"num_write_kb":303552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002391"},"pg_stats":[{"pgid":"4.f","version":"58'636","reported_seq":649,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:48.174095+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:14:48.174095+0000","last_peered":"2024-09-22T03:14:48.174095+0000","last_clean":"2024-09-22T03:14:48.174095+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:14:48.174095+0000","last_undegraded":"2024-09-22T03:14:48.174095+0000","last_fullsized":"2024-09-22T03:14:48.174095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41746432,"num_objects":637,"num_object_clones":0,"num_object_copies":1911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":637,"num_write_kb":40768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"58'663","reported_seq":676,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:48.251275+0000","last_change":"2024-09-22T03:14:21.195156+0000","last_active":"2024-09-22T03:14:48.251275+0000","last_peered":"2024-09-22T03:14:48.251275+0000","last_clean":"2024-09-22T03:14:48.251275+0000","last_became_active":"2024-09-22T03:14:21.188092+0000","last_became_peered":"2024-09-22T03:14:21.188092+0000","last_unstale":"2024-09-22T03:14:48.251275+0000","last_undegraded":"2024-09-22T03:14:48.251275+0000","last_fullsized":"2024-09-22T03:14:48.251275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43515904,"num_objects":664,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":42496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"58'707","reported_seq":720,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.205651+0000","last_change":"2024-09-22T03:14:21.204545+0000","last_active":"2024-09-22T03:14:50.205651+0000","last_peered":"2024-09-22T03:14:50.205651+0000","last_clean":"2024-09-22T03:14:50.205651+0000","last_became_active":"2024-09-22T03:14:21.197364+0000","last_became_peered":"2024-09-22T03:14:21.197364+0000","last_unstale":"2024-09-22T03:14:50.205651+0000","last_undegraded":"2024-09-22T03:14:50.205651+0000","last_fullsized":"2024-09-22T03:14:50.205651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46399488,"num_objects":708,"num_object_clones":0,"num_object_copies":2124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":708,"num_write_kb":45312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"58'691","reported_seq":704,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:51.038090+0000","last_change":"2024-09-22T03:14:21.194514+0000","last_active":"2024-09-22T03:14:51.038090+0000","last_peered":"2024-09-22T03:14:51.038090+0000","last_clean":"2024-09-22T03:14:51.038090+0000","last_became_active":"2024-09-22T03:14:21.184663+0000","last_became_peered":"2024-09-22T03:14:21.184663+0000","last_unstale":"2024-09-22T03:14:51.038090+0000","last_undegraded":"2024-09-22T03:14:51.038090+0000","last_fullsized":"2024-09-22T03:14:51.038090+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45350912,"num_objects":692,"num_object_clones":0,"num_object_copies":2076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":692,"num_write_kb":44288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"58'626","reported_seq":639,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:47.191610+0000","last_change":"2024-09-22T03:14:21.192411+0000","last_active":"2024-09-22T03:14:47.191610+0000","last_peered":"2024-09-22T03:14:47.191610+0000","last_clean":"2024-09-22T03:14:47.191610+0000","last_became_active":"2024-09-22T03:14:21.188971+0000","last_became_peered":"2024-09-22T03:14:21.188971+0000","last_unstale":"2024-09-22T03:14:47.191610+0000","last_undegraded":"2024-09-22T03:14:47.191610+0000","last_fullsized":"2024-09-22T03:14:47.191610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41091072,"num_objects":627,"num_object_clones":0,"num_object_copies":1881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":627,"num_write_kb":40128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"58'722","reported_seq":735,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.201624+0000","last_change":"2024-09-22T03:14:21.197231+0000","last_active":"2024-09-22T03:14:50.201624+0000","last_peered":"2024-09-22T03:14:50.201624+0000","last_clean":"2024-09-22T03:14:50.201624+0000","last_became_active":"2024-09-22T03:14:21.189148+0000","last_became_peered":"2024-09-22T03:14:21.189148+0000","last_unstale":"2024-09-22T03:14:50.201624+0000","last_undegraded":"2024-09-22T03:14:50.201624+0000","last_fullsized":"2024-09-22T03:14:50.201624+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47382528,"num_objects":723,"num_object_clones":0,"num_object_copies":2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723,"num_write_kb":46272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"58'701","reported_seq":714,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.311314+0000","last_change":"2024-09-22T03:14:21.221221+0000","last_active":"2024-09-22T03:14:50.311314+0000","last_peered":"2024-09-22T03:14:50.311314+0000","last_clean":"2024-09-22T03:14:50.311314+0000","last_became_active":"2024-09-22T03:14:21.215886+0000","last_became_peered":"2024-09-22T03:14:21.215886+0000","last_unstale":"2024-09-22T03:14:50.311314+0000","last_undegraded":"2024-09-22T03:14:50.311314+0000","last_fullsized":"2024-09-22T03:14:50.311314+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46006272,"num_objects":702,"num_object_clones":0,"num_object_copies":2106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":702,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"58'675","reported_seq":688,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.267474+0000","last_change":"2024-09-22T03:14:21.221882+0000","last_active":"2024-09-22T03:14:50.267474+0000","last_peered":"2024-09-22T03:14:50.267474+0000","last_clean":"2024-09-22T03:14:50.267474+0000","last_became_active":"2024-09-22T03:14:21.217925+0000","last_became_peered":"2024-09-22T03:14:21.217925+0000","last_unstale":"2024-09-22T03:14:50.267474+0000","last_undegraded":"2024-09-22T03:14:50.267474+0000","last_fullsized":"2024-09-22T03:14:50.267474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44302336,"num_objects":676,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":43264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"58'687","reported_seq":700,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.274837+0000","last_change":"2024-09-22T03:14:21.192905+0000","last_active":"2024-09-22T03:14:50.274837+0000","last_peered":"2024-09-22T03:14:50.274837+0000","last_clean":"2024-09-22T03:14:50.274837+0000","last_became_active":"2024-09-22T03:14:21.185129+0000","last_became_peered":"2024-09-22T03:14:21.185129+0000","last_unstale":"2024-09-22T03:14:50.274837+0000","last_undegraded":"2024-09-22T03:14:50.274837+0000","last_fullsized":"2024-09-22T03:14:50.274837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45088768,"num_objects":688,"num_object_clones":0,"num_object_copies":2064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":44032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.128985+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:14:22.128985+0000","last_peered":"2024-09-22T03:14:22.128985+0000","last_clean":"2024-09-22T03:14:22.128985+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:14:22.128985+0000","last_undegraded":"2024-09-22T03:14:22.128985+0000","last_fullsized":"2024-09-22T03:14:22.128985+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"58'660","reported_seq":673,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:48.153563+0000","last_change":"2024-09-22T03:14:21.206700+0000","last_active":"2024-09-22T03:14:48.153563+0000","last_peered":"2024-09-22T03:14:48.153563+0000","last_clean":"2024-09-22T03:14:48.153563+0000","last_became_active":"2024-09-22T03:14:21.202903+0000","last_became_peered":"2024-09-22T03:14:21.202903+0000","last_unstale":"2024-09-22T03:14:48.153563+0000","last_undegraded":"2024-09-22T03:14:48.153563+0000","last_fullsized":"2024-09-22T03:14:48.153563+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43319296,"num_objects":661,"num_object_clones":0,"num_object_copies":1983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":661,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":155,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.123583+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:14:22.123583+0000","last_peered":"2024-09-22T03:14:22.123583+0000","last_clean":"2024-09-22T03:14:22.123583+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:14:22.123583+0000","last_undegraded":"2024-09-22T03:14:22.123583+0000","last_fullsized":"2024-09-22T03:14:22.123583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"58'640","reported_seq":653,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:48.184725+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:14:48.184725+0000","last_peered":"2024-09-22T03:14:48.184725+0000","last_clean":"2024-09-22T03:14:48.184725+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:14:48.184725+0000","last_undegraded":"2024-09-22T03:14:48.184725+0000","last_fullsized":"2024-09-22T03:14:48.184725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42008576,"num_objects":641,"num_object_clones":0,"num_object_copies":1923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":641,"num_write_kb":41024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.126298+0000","last_change":"2024-09-22T03:14:19.140598+0000","last_active":"2024-09-22T03:14:22.126298+0000","last_peered":"2024-09-22T03:14:22.126298+0000","last_clean":"2024-09-22T03:14:22.126298+0000","last_became_active":"2024-09-22T03:14:07.191763+0000","last_became_peered":"2024-09-22T03:14:07.191763+0000","last_unstale":"2024-09-22T03:14:22.126298+0000","last_undegraded":"2024-09-22T03:14:22.126298+0000","last_fullsized":"2024-09-22T03:14:22.126298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"58'630","reported_seq":643,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:47.139186+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:14:47.139186+0000","last_peered":"2024-09-22T03:14:47.139186+0000","last_clean":"2024-09-22T03:14:47.139186+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:14:47.139186+0000","last_undegraded":"2024-09-22T03:14:47.139186+0000","last_fullsized":"2024-09-22T03:14:47.139186+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41353216,"num_objects":631,"num_object_clones":0,"num_object_copies":1893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":631,"num_write_kb":40384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129978+0000","last_change":"2024-09-22T03:14:19.157303+0000","last_active":"2024-09-22T03:14:22.129978+0000","last_peered":"2024-09-22T03:14:22.129978+0000","last_clean":"2024-09-22T03:14:22.129978+0000","last_became_active":"2024-09-22T03:14:07.230230+0000","last_became_peered":"2024-09-22T03:14:07.230230+0000","last_unstale":"2024-09-22T03:14:22.129978+0000","last_undegraded":"2024-09-22T03:14:22.129978+0000","last_fullsized":"2024-09-22T03:14:22.129978+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":279,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129224+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:14:22.129224+0000","last_peered":"2024-09-22T03:14:22.129224+0000","last_clean":"2024-09-22T03:14:22.129224+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:14:22.129224+0000","last_undegraded":"2024-09-22T03:14:22.129224+0000","last_fullsized":"2024-09-22T03:14:22.129224+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"58'699","reported_seq":712,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.306224+0000","last_change":"2024-09-22T03:14:21.179859+0000","last_active":"2024-09-22T03:14:50.306224+0000","last_peered":"2024-09-22T03:14:50.306224+0000","last_clean":"2024-09-22T03:14:50.306224+0000","last_became_active":"2024-09-22T03:14:21.175053+0000","last_became_peered":"2024-09-22T03:14:21.175053+0000","last_unstale":"2024-09-22T03:14:50.306224+0000","last_undegraded":"2024-09-22T03:14:50.306224+0000","last_fullsized":"2024-09-22T03:14:50.306224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45875200,"num_objects":700,"num_object_clones":0,"num_object_copies":2100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":700,"num_write_kb":44800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.127690+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:14:22.127690+0000","last_peered":"2024-09-22T03:14:22.127690+0000","last_clean":"2024-09-22T03:14:22.127690+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:14:22.127690+0000","last_undegraded":"2024-09-22T03:14:22.127690+0000","last_fullsized":"2024-09-22T03:14:22.127690+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"58'591","reported_seq":604,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:47.167799+0000","last_change":"2024-09-22T03:14:21.200985+0000","last_active":"2024-09-22T03:14:47.167799+0000","last_peered":"2024-09-22T03:14:47.167799+0000","last_clean":"2024-09-22T03:14:47.167799+0000","last_became_active":"2024-09-22T03:14:21.196935+0000","last_became_peered":"2024-09-22T03:14:21.196935+0000","last_unstale":"2024-09-22T03:14:47.167799+0000","last_undegraded":"2024-09-22T03:14:47.167799+0000","last_fullsized":"2024-09-22T03:14:47.167799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38797312,"num_objects":592,"num_object_clones":0,"num_object_copies":1776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":37888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130987+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:14:22.130987+0000","last_peered":"2024-09-22T03:14:22.130987+0000","last_clean":"2024-09-22T03:14:22.130987+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:14:22.130987+0000","last_undegraded":"2024-09-22T03:14:22.130987+0000","last_fullsized":"2024-09-22T03:14:22.130987+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"58'650","reported_seq":663,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:47.075107+0000","last_change":"2024-09-22T03:14:21.201433+0000","last_active":"2024-09-22T03:14:47.075107+0000","last_peered":"2024-09-22T03:14:47.075107+0000","last_clean":"2024-09-22T03:14:47.075107+0000","last_became_active":"2024-09-22T03:14:21.196689+0000","last_became_peered":"2024-09-22T03:14:21.196689+0000","last_unstale":"2024-09-22T03:14:47.075107+0000","last_undegraded":"2024-09-22T03:14:47.075107+0000","last_fullsized":"2024-09-22T03:14:47.075107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42663936,"num_objects":651,"num_object_clones":0,"num_object_copies":1953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":651,"num_write_kb":41664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129348+0000","last_change":"2024-09-22T03:14:19.146918+0000","last_active":"2024-09-22T03:14:22.129348+0000","last_peered":"2024-09-22T03:14:22.129348+0000","last_clean":"2024-09-22T03:14:22.129348+0000","last_became_active":"2024-09-22T03:14:07.194059+0000","last_became_peered":"2024-09-22T03:14:07.194059+0000","last_unstale":"2024-09-22T03:14:22.129348+0000","last_undegraded":"2024-09-22T03:14:22.129348+0000","last_fullsized":"2024-09-22T03:14:22.129348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"58'705","reported_seq":718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.261913+0000","last_change":"2024-09-22T03:14:21.222937+0000","last_active":"2024-09-22T03:14:50.261913+0000","last_peered":"2024-09-22T03:14:50.261913+0000","last_clean":"2024-09-22T03:14:50.261913+0000","last_became_active":"2024-09-22T03:14:21.218857+0000","last_became_peered":"2024-09-22T03:14:21.218857+0000","last_unstale":"2024-09-22T03:14:50.261913+0000","last_undegraded":"2024-09-22T03:14:50.261913+0000","last_fullsized":"2024-09-22T03:14:50.261913+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46268416,"num_objects":706,"num_object_clones":0,"num_object_copies":2118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":706,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130197+0000","last_change":"2024-09-22T03:14:19.146650+0000","last_active":"2024-09-22T03:14:22.130197+0000","last_peered":"2024-09-22T03:14:22.130197+0000","last_clean":"2024-09-22T03:14:22.130197+0000","last_became_active":"2024-09-22T03:14:07.200969+0000","last_became_peered":"2024-09-22T03:14:07.200969+0000","last_unstale":"2024-09-22T03:14:22.130197+0000","last_undegraded":"2024-09-22T03:14:22.130197+0000","last_fullsized":"2024-09-22T03:14:22.130197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":701169664,"num_objects":10699,"num_object_clones":0,"num_object_copies":32097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10699,"num_write_kb":684736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":132493312,"data_stored":2119892992,"data_compressed":100660533,"data_compressed_allocated":132493312,"data_compressed_original":2119892992,"omap_allocated":0,"internal_metadata":0},"log_size":10683,"ondisk_log_size":10683,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266732,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75012,"kb_used_data":36268,"kb_used_omap":0,"kb_used_meta":38720,"kb_avail":94296828,"statfs":{"total":96636764160,"available":96559951872,"internally_reserved":0,"allocated":37138432,"data_stored":586601428,"data_compressed":27845284,"data_compressed_allocated":36651008,"data_compressed_original":586416128,"omap_allocated":0,"internal_metadata":39649280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97288,"kb_used_data":26036,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94274552,"statfs":{"total":96636764160,"available":96537141248,"internally_reserved":0,"allocated":26660864,"data_stored":418567668,"data_compressed":19868271,"data_compressed_allocated":26152960,"data_compressed_original":418365440,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100244,"kb_used_data":36676,"kb_used_omap":0,"kb_used_meta":63552,"kb_avail":94271596,"statfs":{"total":96636764160,"available":96534114304,"internally_reserved":0,"allocated":37556224,"data_stored":592893428,"data_compressed":28145991,"data_compressed_allocated":37048320,"data_compressed_original":592691200,"omap_allocated":0,"internal_metadata":65077248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100604,"kb_used_data":29356,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94271236,"statfs":{"total":96636764160,"available":96533745664,"internally_reserved":0,"allocated":30060544,"data_stored":473028065,"data_compressed":22454208,"data_compressed_allocated":29556736,"data_compressed_original":472825856,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30572544,"data_stored":489160704,"data_compressed":23227132,"data_compressed_allocated":30572544,"data_compressed_original":489160704,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37167104,"data_stored":594673664,"data_compressed":28237387,"data_compressed_allocated":37167104,"data_compressed_original":594673664,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26873856,"data_stored":429981696,"data_compressed":20417131,"data_compressed_allocated":26873856,"data_compressed_original":429981696,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37879808,"data_stored":606076928,"data_compressed":28778883,"data_compressed_allocated":37879808,"data_compressed_original":606076928,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:14:52.030 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:52.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:52.179+0000 7f1f56d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:52.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:52.187+0000 7f1f56d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:52.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 30 16 11822 11806 24.5938 21.625 0.0740104 0.0406186 2024-09-22T03:14:53.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 31 16 12190 12174 24.5423 23 0.0323393 0.0406997 2024-09-22T03:14:54.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 32 16 12566 12550 24.5097 23.5 0.0305696 0.0407681 2024-09-22T03:14:54.430 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:54.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:54.575+0000 7f07ad42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:54.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:54.582+0000 7f07ad42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:55.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 33 16 12918 12902 24.4336 22 0.0622128 0.0408804 2024-09-22T03:14:56.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 34 16 13276 13260 24.373 22.375 0.0428916 0.0409998 2024-09-22T03:14:56.453 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:14:56.453 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:14:56.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:14:56.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:56.605+0000 7f605bad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:56.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:56.619+0000 7f605bad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:56.762 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:14:56.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:56.919+0000 7f1587997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:56.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:56.928+0000 7f1587997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:56.947 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:14:56.947 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:14:56.969 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-22T03:14:55.268722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":762774067,"num_objects":11634,"num_object_clones":0,"num_object_copies":34902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11769,"num_write_kb":746216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11714,"ondisk_log_size":11714,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":391552,"kb_used_data":138424,"kb_used_omap":0,"kb_used_meta":253056,"kb_avail":377095808,"statfs":{"total":386547056640,"available":386146107392,"internally_reserved":0,"allocated":141746176,"data_stored":2236372381,"data_compressed":106162216,"data_compressed_allocated":139739136,"data_compressed_original":2235580416,"omap_allocated":0,"internal_metadata":259129344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":275120128,"num_objects":4198,"num_object_clones":0,"num_object_copies":12594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":268672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002295"},"pg_stats":[{"pgid":"4.f","version":"58'752","reported_seq":765,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:53.188457+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:14:53.188457+0000","last_peered":"2024-09-22T03:14:53.188457+0000","last_clean":"2024-09-22T03:14:53.188457+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:14:53.188457+0000","last_undegraded":"2024-09-22T03:14:53.188457+0000","last_fullsized":"2024-09-22T03:14:53.188457+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49348608,"num_objects":753,"num_object_clones":0,"num_object_copies":2259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":753,"num_write_kb":48192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"58'777","reported_seq":790,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:53.255236+0000","last_change":"2024-09-22T03:14:21.195156+0000","last_active":"2024-09-22T03:14:53.255236+0000","last_peered":"2024-09-22T03:14:53.255236+0000","last_clean":"2024-09-22T03:14:53.255236+0000","last_became_active":"2024-09-22T03:14:21.188092+0000","last_became_peered":"2024-09-22T03:14:21.188092+0000","last_unstale":"2024-09-22T03:14:53.255236+0000","last_undegraded":"2024-09-22T03:14:53.255236+0000","last_fullsized":"2024-09-22T03:14:53.255236+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50987008,"num_objects":778,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":778,"num_write_kb":49792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"58'707","reported_seq":720,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.205651+0000","last_change":"2024-09-22T03:14:21.204545+0000","last_active":"2024-09-22T03:14:50.205651+0000","last_peered":"2024-09-22T03:14:50.205651+0000","last_clean":"2024-09-22T03:14:50.205651+0000","last_became_active":"2024-09-22T03:14:21.197364+0000","last_became_peered":"2024-09-22T03:14:21.197364+0000","last_unstale":"2024-09-22T03:14:50.205651+0000","last_undegraded":"2024-09-22T03:14:50.205651+0000","last_fullsized":"2024-09-22T03:14:50.205651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46399488,"num_objects":708,"num_object_clones":0,"num_object_copies":2124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":708,"num_write_kb":45312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"58'691","reported_seq":704,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:51.038090+0000","last_change":"2024-09-22T03:14:21.194514+0000","last_active":"2024-09-22T03:14:51.038090+0000","last_peered":"2024-09-22T03:14:51.038090+0000","last_clean":"2024-09-22T03:14:51.038090+0000","last_became_active":"2024-09-22T03:14:21.184663+0000","last_became_peered":"2024-09-22T03:14:21.184663+0000","last_unstale":"2024-09-22T03:14:51.038090+0000","last_undegraded":"2024-09-22T03:14:51.038090+0000","last_fullsized":"2024-09-22T03:14:51.038090+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45350912,"num_objects":692,"num_object_clones":0,"num_object_copies":2076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":692,"num_write_kb":44288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"58'739","reported_seq":752,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:52.167368+0000","last_change":"2024-09-22T03:14:21.192411+0000","last_active":"2024-09-22T03:14:52.167368+0000","last_peered":"2024-09-22T03:14:52.167368+0000","last_clean":"2024-09-22T03:14:52.167368+0000","last_became_active":"2024-09-22T03:14:21.188971+0000","last_became_peered":"2024-09-22T03:14:21.188971+0000","last_unstale":"2024-09-22T03:14:52.167368+0000","last_undegraded":"2024-09-22T03:14:52.167368+0000","last_fullsized":"2024-09-22T03:14:52.167368+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48496640,"num_objects":740,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":740,"num_write_kb":47360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"58'722","reported_seq":735,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.201624+0000","last_change":"2024-09-22T03:14:21.197231+0000","last_active":"2024-09-22T03:14:50.201624+0000","last_peered":"2024-09-22T03:14:50.201624+0000","last_clean":"2024-09-22T03:14:50.201624+0000","last_became_active":"2024-09-22T03:14:21.189148+0000","last_became_peered":"2024-09-22T03:14:21.189148+0000","last_unstale":"2024-09-22T03:14:50.201624+0000","last_undegraded":"2024-09-22T03:14:50.201624+0000","last_fullsized":"2024-09-22T03:14:50.201624+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47382528,"num_objects":723,"num_object_clones":0,"num_object_copies":2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723,"num_write_kb":46272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"58'701","reported_seq":714,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.311314+0000","last_change":"2024-09-22T03:14:21.221221+0000","last_active":"2024-09-22T03:14:50.311314+0000","last_peered":"2024-09-22T03:14:50.311314+0000","last_clean":"2024-09-22T03:14:50.311314+0000","last_became_active":"2024-09-22T03:14:21.215886+0000","last_became_peered":"2024-09-22T03:14:21.215886+0000","last_unstale":"2024-09-22T03:14:50.311314+0000","last_undegraded":"2024-09-22T03:14:50.311314+0000","last_fullsized":"2024-09-22T03:14:50.311314+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46006272,"num_objects":702,"num_object_clones":0,"num_object_copies":2106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":702,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"58'675","reported_seq":688,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.267474+0000","last_change":"2024-09-22T03:14:21.221882+0000","last_active":"2024-09-22T03:14:50.267474+0000","last_peered":"2024-09-22T03:14:50.267474+0000","last_clean":"2024-09-22T03:14:50.267474+0000","last_became_active":"2024-09-22T03:14:21.217925+0000","last_became_peered":"2024-09-22T03:14:21.217925+0000","last_unstale":"2024-09-22T03:14:50.267474+0000","last_undegraded":"2024-09-22T03:14:50.267474+0000","last_fullsized":"2024-09-22T03:14:50.267474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44302336,"num_objects":676,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":43264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"58'687","reported_seq":700,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.274837+0000","last_change":"2024-09-22T03:14:21.192905+0000","last_active":"2024-09-22T03:14:50.274837+0000","last_peered":"2024-09-22T03:14:50.274837+0000","last_clean":"2024-09-22T03:14:50.274837+0000","last_became_active":"2024-09-22T03:14:21.185129+0000","last_became_peered":"2024-09-22T03:14:21.185129+0000","last_unstale":"2024-09-22T03:14:50.274837+0000","last_undegraded":"2024-09-22T03:14:50.274837+0000","last_fullsized":"2024-09-22T03:14:50.274837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45088768,"num_objects":688,"num_object_clones":0,"num_object_copies":2064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":44032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.128985+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:14:22.128985+0000","last_peered":"2024-09-22T03:14:22.128985+0000","last_clean":"2024-09-22T03:14:22.128985+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:14:22.128985+0000","last_undegraded":"2024-09-22T03:14:22.128985+0000","last_fullsized":"2024-09-22T03:14:22.128985+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"58'788","reported_seq":801,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:53.238389+0000","last_change":"2024-09-22T03:14:21.206700+0000","last_active":"2024-09-22T03:14:53.238389+0000","last_peered":"2024-09-22T03:14:53.238389+0000","last_clean":"2024-09-22T03:14:53.238389+0000","last_became_active":"2024-09-22T03:14:21.202903+0000","last_became_peered":"2024-09-22T03:14:21.202903+0000","last_unstale":"2024-09-22T03:14:53.238389+0000","last_undegraded":"2024-09-22T03:14:53.238389+0000","last_fullsized":"2024-09-22T03:14:53.238389+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51707904,"num_objects":789,"num_object_clones":0,"num_object_copies":2367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":789,"num_write_kb":50496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":155,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.123583+0000","last_change":"2024-09-22T03:12:32.024100+0000","last_active":"2024-09-22T03:14:22.123583+0000","last_peered":"2024-09-22T03:14:22.123583+0000","last_clean":"2024-09-22T03:14:22.123583+0000","last_became_active":"2024-09-22T03:12:31.080148+0000","last_became_peered":"2024-09-22T03:12:31.080148+0000","last_unstale":"2024-09-22T03:14:22.123583+0000","last_undegraded":"2024-09-22T03:14:22.123583+0000","last_fullsized":"2024-09-22T03:14:22.123583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"58'760","reported_seq":773,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:53.135090+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:14:53.135090+0000","last_peered":"2024-09-22T03:14:53.135090+0000","last_clean":"2024-09-22T03:14:53.135090+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:14:53.135090+0000","last_undegraded":"2024-09-22T03:14:53.135090+0000","last_fullsized":"2024-09-22T03:14:53.135090+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49872896,"num_objects":761,"num_object_clones":0,"num_object_copies":2283,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":48704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.126298+0000","last_change":"2024-09-22T03:14:19.140598+0000","last_active":"2024-09-22T03:14:22.126298+0000","last_peered":"2024-09-22T03:14:22.126298+0000","last_clean":"2024-09-22T03:14:22.126298+0000","last_became_active":"2024-09-22T03:14:07.191763+0000","last_became_peered":"2024-09-22T03:14:07.191763+0000","last_unstale":"2024-09-22T03:14:22.126298+0000","last_undegraded":"2024-09-22T03:14:22.126298+0000","last_fullsized":"2024-09-22T03:14:22.126298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"58'735","reported_seq":748,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:52.103526+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:14:52.103526+0000","last_peered":"2024-09-22T03:14:52.103526+0000","last_clean":"2024-09-22T03:14:52.103526+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:14:52.103526+0000","last_undegraded":"2024-09-22T03:14:52.103526+0000","last_fullsized":"2024-09-22T03:14:52.103526+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129978+0000","last_change":"2024-09-22T03:14:19.157303+0000","last_active":"2024-09-22T03:14:22.129978+0000","last_peered":"2024-09-22T03:14:22.129978+0000","last_clean":"2024-09-22T03:14:22.129978+0000","last_became_active":"2024-09-22T03:14:07.230230+0000","last_became_peered":"2024-09-22T03:14:07.230230+0000","last_unstale":"2024-09-22T03:14:22.129978+0000","last_undegraded":"2024-09-22T03:14:22.129978+0000","last_fullsized":"2024-09-22T03:14:22.129978+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":279,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129224+0000","last_change":"2024-09-22T03:12:31.098730+0000","last_active":"2024-09-22T03:14:22.129224+0000","last_peered":"2024-09-22T03:14:22.129224+0000","last_clean":"2024-09-22T03:14:22.129224+0000","last_became_active":"2024-09-22T03:12:31.092027+0000","last_became_peered":"2024-09-22T03:12:31.092027+0000","last_unstale":"2024-09-22T03:14:22.129224+0000","last_undegraded":"2024-09-22T03:14:22.129224+0000","last_fullsized":"2024-09-22T03:14:22.129224+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"58'699","reported_seq":712,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.306224+0000","last_change":"2024-09-22T03:14:21.179859+0000","last_active":"2024-09-22T03:14:50.306224+0000","last_peered":"2024-09-22T03:14:50.306224+0000","last_clean":"2024-09-22T03:14:50.306224+0000","last_became_active":"2024-09-22T03:14:21.175053+0000","last_became_peered":"2024-09-22T03:14:21.175053+0000","last_unstale":"2024-09-22T03:14:50.306224+0000","last_undegraded":"2024-09-22T03:14:50.306224+0000","last_fullsized":"2024-09-22T03:14:50.306224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45875200,"num_objects":700,"num_object_clones":0,"num_object_copies":2100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":700,"num_write_kb":44800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.127690+0000","last_change":"2024-09-22T03:12:32.028732+0000","last_active":"2024-09-22T03:14:22.127690+0000","last_peered":"2024-09-22T03:14:22.127690+0000","last_clean":"2024-09-22T03:14:22.127690+0000","last_became_active":"2024-09-22T03:12:31.088092+0000","last_became_peered":"2024-09-22T03:12:31.088092+0000","last_unstale":"2024-09-22T03:14:22.127690+0000","last_undegraded":"2024-09-22T03:14:22.127690+0000","last_fullsized":"2024-09-22T03:14:22.127690+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"58'718","reported_seq":731,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:52.183390+0000","last_change":"2024-09-22T03:14:21.200985+0000","last_active":"2024-09-22T03:14:52.183390+0000","last_peered":"2024-09-22T03:14:52.183390+0000","last_clean":"2024-09-22T03:14:52.183390+0000","last_became_active":"2024-09-22T03:14:21.196935+0000","last_became_peered":"2024-09-22T03:14:21.196935+0000","last_unstale":"2024-09-22T03:14:52.183390+0000","last_undegraded":"2024-09-22T03:14:52.183390+0000","last_fullsized":"2024-09-22T03:14:52.183390+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130987+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:14:22.130987+0000","last_peered":"2024-09-22T03:14:22.130987+0000","last_clean":"2024-09-22T03:14:22.130987+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:14:22.130987+0000","last_undegraded":"2024-09-22T03:14:22.130987+0000","last_fullsized":"2024-09-22T03:14:22.130987+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"58'758","reported_seq":771,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:52.158504+0000","last_change":"2024-09-22T03:14:21.201433+0000","last_active":"2024-09-22T03:14:52.158504+0000","last_peered":"2024-09-22T03:14:52.158504+0000","last_clean":"2024-09-22T03:14:52.158504+0000","last_became_active":"2024-09-22T03:14:21.196689+0000","last_became_peered":"2024-09-22T03:14:21.196689+0000","last_unstale":"2024-09-22T03:14:52.158504+0000","last_undegraded":"2024-09-22T03:14:52.158504+0000","last_fullsized":"2024-09-22T03:14:52.158504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49741824,"num_objects":759,"num_object_clones":0,"num_object_copies":2277,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":759,"num_write_kb":48576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.129348+0000","last_change":"2024-09-22T03:14:19.146918+0000","last_active":"2024-09-22T03:14:22.129348+0000","last_peered":"2024-09-22T03:14:22.129348+0000","last_clean":"2024-09-22T03:14:22.129348+0000","last_became_active":"2024-09-22T03:14:07.194059+0000","last_became_peered":"2024-09-22T03:14:07.194059+0000","last_unstale":"2024-09-22T03:14:22.129348+0000","last_undegraded":"2024-09-22T03:14:22.129348+0000","last_fullsized":"2024-09-22T03:14:22.129348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"58'705","reported_seq":718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:50.261913+0000","last_change":"2024-09-22T03:14:21.222937+0000","last_active":"2024-09-22T03:14:50.261913+0000","last_peered":"2024-09-22T03:14:50.261913+0000","last_clean":"2024-09-22T03:14:50.261913+0000","last_became_active":"2024-09-22T03:14:21.218857+0000","last_became_peered":"2024-09-22T03:14:21.218857+0000","last_unstale":"2024-09-22T03:14:50.261913+0000","last_undegraded":"2024-09-22T03:14:50.261913+0000","last_fullsized":"2024-09-22T03:14:50.261913+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46268416,"num_objects":706,"num_object_clones":0,"num_object_copies":2118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":706,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:14:22.130197+0000","last_change":"2024-09-22T03:14:19.146650+0000","last_active":"2024-09-22T03:14:22.130197+0000","last_peered":"2024-09-22T03:14:22.130197+0000","last_clean":"2024-09-22T03:14:22.130197+0000","last_became_active":"2024-09-22T03:14:07.200969+0000","last_became_peered":"2024-09-22T03:14:07.200969+0000","last_unstale":"2024-09-22T03:14:22.130197+0000","last_undegraded":"2024-09-22T03:14:22.130197+0000","last_fullsized":"2024-09-22T03:14:22.130197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":762183680,"num_objects":11630,"num_object_clones":0,"num_object_copies":34890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11630,"num_write_kb":744320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":142733312,"data_stored":2283732992,"data_compressed":108440531,"data_compressed_allocated":142733312,"data_compressed_original":2283732992,"omap_allocated":0,"internal_metadata":0},"log_size":11614,"ondisk_log_size":11614,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266732,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75012,"kb_used_data":36268,"kb_used_omap":0,"kb_used_meta":38720,"kb_avail":94296828,"statfs":{"total":96636764160,"available":96559951872,"internally_reserved":0,"allocated":37138432,"data_stored":586601428,"data_compressed":27845284,"data_compressed_allocated":36651008,"data_compressed_original":586416128,"omap_allocated":0,"internal_metadata":39649280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106032,"kb_used_data":30624,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94265808,"statfs":{"total":96636764160,"available":96528187392,"internally_reserved":0,"allocated":31358976,"data_stored":493737460,"data_compressed":23437734,"data_compressed_allocated":30851072,"data_compressed_original":493535232,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":26,"seq":111669149836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100244,"kb_used_data":36676,"kb_used_omap":0,"kb_used_meta":63552,"kb_avail":94271596,"statfs":{"total":96636764160,"available":96534114304,"internally_reserved":0,"allocated":37556224,"data_stored":592893428,"data_compressed":28145991,"data_compressed_allocated":37048320,"data_compressed_original":592691200,"omap_allocated":0,"internal_metadata":65077248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110264,"kb_used_data":34856,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94261576,"statfs":{"total":96636764160,"available":96523853824,"internally_reserved":0,"allocated":35692544,"data_stored":563140065,"data_compressed":26733207,"data_compressed_allocated":35188736,"data_compressed_original":562937856,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36077568,"data_stored":577241088,"data_compressed":27409659,"data_compressed_allocated":36077568,"data_compressed_original":577241088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37167104,"data_stored":594673664,"data_compressed":28237387,"data_compressed_allocated":37167104,"data_compressed_original":594673664,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31608832,"data_stored":505741312,"data_compressed":24014602,"data_compressed_allocated":31608832,"data_compressed_original":505741312,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37879808,"data_stored":606076928,"data_compressed":28778883,"data_compressed_allocated":37879808,"data_compressed_original":606076928,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:14:57.203 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 35 16 13616 13600 24.2837 21.25 0.0644427 0.0411542 2024-09-22T03:14:58.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 36 16 13952 13936 24.1925 21 0.0713101 0.0412974 2024-09-22T03:14:59.101 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:14:59.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 37 16 14301 14285 24.1281 21.8125 0.0763612 0.0414068 2024-09-22T03:14:59.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:59.242+0000 7f3c42d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:14:59.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:14:59.251+0000 7f3c42d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:00.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 38 16 14630 14614 24.0342 20.5625 0.0347603 0.0415663 2024-09-22T03:15:01.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 39 16 14988 14972 23.9916 22.375 0.0392123 0.0416534 2024-09-22T03:15:01.439 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:15:01.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:01.594+0000 7ffab8b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:01.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:01.602+0000 7ffab8b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:01.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:15:01.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:15:01.971 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:15:01.971 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:15:02.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.108+0000 7f4925c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.118+0000 7f4925c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.148 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:02.149 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:02.150 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:02.150 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:02.150 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:02.150 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:02.150 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:02.151 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:02.151 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:02.151 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:02.151 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:02.151 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:02.156 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:02.156 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:02.156 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:15:02.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:15:02.203520+0000 min lat: 0.00928913 max lat: 0.1236 avg lat: 0.0418153 2024-09-22T03:15:02.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:15:02.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 40 16 15310 15294 23.8949 20.125 0.0309523 0.0418153 2024-09-22T03:15:02.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.296+0000 7fea26917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.313+0000 7fea26917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.415 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:02.416 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:02.417 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:02.417 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:02.417 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:02.419 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:02.419 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:02.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:15:02.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.569+0000 7f5254c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.586+0000 7f5254c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.609 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:02.609 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:02.614 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:02.614 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:02.615 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:15:02.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.762+0000 7fe02d463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:02.777+0000 7fe02d463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:02.848 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:02.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:02.852 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:03.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 41 16 15649 15633 23.8288 21.1875 0.0614529 0.0419426 2024-09-22T03:15:03.857 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:15:04.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:04.019+0000 7fd684397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:04.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:04.027+0000 7fd684397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:04.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 42 16 15987 15971 23.7644 21.125 0.0303275 0.0420501 2024-09-22T03:15:05.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 43 16 16317 16301 23.6914 20.625 0.0223311 0.0421673 2024-09-22T03:15:06.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 44 16 16658 16642 23.6373 21.3125 0.0400441 0.042273 2024-09-22T03:15:06.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:15:06.415 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:06.411+0000 7f2c034d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:06.419+0000 7f2c034d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:07.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 45 16 16992 16976 23.5758 20.875 0.0683272 0.0423893 2024-09-22T03:15:07.853 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-22T03:15:07.854 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:15:07.854 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 1, 3] 2024-09-22T03:15:07.854 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:15:07.854 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:15:08.204 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 46 16 17225 17209 23.3799 14.5625 0.0281052 0.0424385 2024-09-22T03:15:08.639 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:15:08.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:15:08.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:08.739+0000 7f4c70b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:08.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:08.743+0000 7f4c70b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:09.205 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 47 16 17225 17209 22.8824 0 - 0.0424385 2024-09-22T03:15:10.205 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 48 16 17225 17209 22.4057 0 - 0.0424385 2024-09-22T03:15:11.205 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 49 16 17225 17209 21.9483 0 - 0.0424385 2024-09-22T03:15:12.205 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 50 16 17225 17209 21.5094 0 - 0.0424385 2024-09-22T03:15:13.205 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 51 16 17225 17209 21.0876 0 - 0.0424385 2024-09-22T03:15:13.955 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:15:14.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~6s 2024-09-22T03:15:14.205 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 52 16 17225 17209 20.6821 0 - 0.0424385 2024-09-22T03:15:15.205 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 53 16 17225 17209 20.2918 0 - 0.0424385 2024-09-22T03:15:16.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 54 16 17225 17209 19.916 0 - 0.0424385 2024-09-22T03:15:17.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 55 16 17225 17209 19.5539 0 - 0.0424385 2024-09-22T03:15:18.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 56 16 17225 17209 19.2047 0 - 0.0424385 2024-09-22T03:15:18.956 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-22T03:15:18.956 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:15:18.956 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:15:18.956 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:15:18.956 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:15:18.959 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:15:18.959 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:15:19.071 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:15:19.074 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:19.074 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:15:19.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 57 16 17225 17209 18.8678 0 - 0.0424385 2024-09-22T03:15:19.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:15:20.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 58 16 17225 17209 18.5425 0 - 0.0424385 2024-09-22T03:15:21.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 59 16 17225 17209 18.2282 0 - 0.0424385 2024-09-22T03:15:22.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:15:22.205838+0000 min lat: 0.00928913 max lat: 0.1236 avg lat: 0.0424385 2024-09-22T03:15:22.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:15:22.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 60 16 17225 17209 17.9244 0 - 0.0424385 2024-09-22T03:15:23.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 61 16 17225 17209 17.6305 0 - 0.0424385 2024-09-22T03:15:24.075 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:15:24.171 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:15:24.174 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:24.175 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:15:24.206 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 62 16 17225 17209 17.3462 0 - 0.0424385 2024-09-22T03:15:25.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 63 16 17225 17209 17.0708 0 - 0.0424385 2024-09-22T03:15:26.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 64 16 17225 17209 16.8041 0 - 0.0424385 2024-09-22T03:15:27.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 65 16 17225 17209 16.5455 0 - 0.0424385 2024-09-22T03:15:28.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 66 16 17225 17209 16.2948 0 - 0.0424385 2024-09-22T03:15:29.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:15:29.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 67 16 17225 17209 16.0516 0 - 0.0424385 2024-09-22T03:15:29.274 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:15:29.277 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:29.277 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:15:30.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 68 16 17225 17209 15.8156 0 - 0.0424385 2024-09-22T03:15:31.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 69 16 17225 17209 15.5864 0 - 0.0424385 2024-09-22T03:15:32.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 70 16 17225 17209 15.3637 0 - 0.0424385 2024-09-22T03:15:33.207 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 71 16 17225 17209 15.1473 0 - 0.0424385 2024-09-22T03:15:34.046 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:15:34.145 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:15:34.146 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:15:34.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 72 16 17225 17209 14.9369 0 - 0.0424385 2024-09-22T03:15:34.279 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:15:34.378 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:15:34.378 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [] 2024-09-22T03:15:34.379 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:15:35.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 73 16 17225 17209 14.7323 0 - 0.0424385 2024-09-22T03:15:36.146 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:15:36.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 74 16 17230 17214 14.5375 0.0111607 28.3557 0.0506558 2024-09-22T03:15:36.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:36.276+0000 7fdbb5797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:36.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:36.282+0000 7fdbb5797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:37.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 75 16 17525 17509 14.5894 18.4375 0.0601907 0.0685039 2024-09-22T03:15:38.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 76 16 17869 17853 14.6803 21.5 0.0564116 0.0680921 2024-09-22T03:15:38.505 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:15:38.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:38.661+0000 7fed58357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:38.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:38.669+0000 7fed58357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:39.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 77 16 18207 18191 14.764 21.125 0.030054 0.0677018 2024-09-22T03:15:39.388 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-22T03:15:39.388 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:15:39.388 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:15:39.389 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:15:39.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:39.525+0000 7f09f30b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:39.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:39.541+0000 7f09f30b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:39.575 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:39.575 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:39.575 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:39.575 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:39.578 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:39.578 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:39.578 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:39.578 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:39.578 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:39.578 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:39.579 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:39.580 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:39.581 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:39.581 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:15:39.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:39.737+0000 7f9550197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:39.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:39.750+0000 7f9550197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:39.859 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:39.860 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:39.864 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:39.864 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:39.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:15:40.020 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:40.011+0000 7f9a4f8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:40.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:40.024+0000 7f9a4f8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:40.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:40.105 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:40.110 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:40.110 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:40.110 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:15:40.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 78 16 18523 18507 14.8279 19.75 0.0571298 0.0674104 2024-09-22T03:15:40.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:40.257+0000 7f582b917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:40.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:40.269+0000 7f582b917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:40.307 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:40.307 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:40.308 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:40.312 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:40.312 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:41.071 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:15:41.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 79 16 18853 18837 14.9013 20.625 0.0510874 0.0670761 2024-09-22T03:15:41.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:41.229+0000 7fad5fa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:41.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:41.237+0000 7fad5fa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:42.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:15:42.207844+0000 min lat: 0.00928913 max lat: 28.4584 avg lat: 0.0667318 2024-09-22T03:15:42.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:15:42.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 80 16 19192 19176 14.9798 21.1875 0.0359735 0.0667318 2024-09-22T03:15:43.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 81 16 19530 19514 15.0557 21.125 0.0725783 0.0663892 2024-09-22T03:15:43.517 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:15:43.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:43.657+0000 7f9adb597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:43.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:43.664+0000 7f9adb597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:44.208 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 82 16 19854 19838 15.119 20.25 0.0407622 0.0661027 2024-09-22T03:15:45.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 83 16 20186 20170 15.1868 20.75 0.0707712 0.0658259 2024-09-22T03:15:45.312 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-22T03:15:45.313 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:15:45.313 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:15:45.313 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:15:45.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:45.446+0000 7f1190cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:45.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:45.460+0000 7f1190cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:45.485 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:45.486 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:45.486 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:45.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:45.490 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:45.490 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:15:45.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:45.639+0000 7fa8db22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:45.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:45.655+0000 7fa8db22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:45.767 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:45.767 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:45.768 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:45.772 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:45.772 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:45.772 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:15:45.912 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:15:45.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:45.934+0000 7f856eeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:45.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:45.956+0000 7f856eeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:45.996 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:45.996 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:45.996 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:45.996 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:45.996 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:45.996 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:45.997 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:45.997 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:45.997 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:45.997 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:45.997 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:45.997 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:46.000 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:46.000 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:46.001 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:15:46.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:46.069+0000 7ff36d797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:46.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:46.078+0000 7ff367fff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:46.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 84 16 20466 20450 15.2143 17.5 0.059294 0.065696 2024-09-22T03:15:46.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:46.208+0000 7f05d006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:46.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:46.223+0000 7f05d006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:15:46.244 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:15:46.245 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:15:46.249 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:15:46.249 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:15:47.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 85 16 20787 20771 15.2713 20.0625 0.0338953 0.0654576 2024-09-22T03:15:48.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 86 16 21124 21108 15.3386 21.0625 0.0478944 0.065168 2024-09-22T03:15:48.388 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:15:48.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:48.547+0000 7f437062c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:48.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:48.556+0000 7f437062c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:49.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 87 16 21440 21424 15.3893 19.75 0.0580864 0.0649546 2024-09-22T03:15:50.209 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 88 16 21770 21754 15.4488 20.625 0.0461306 0.0647063 2024-09-22T03:15:50.872 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:15:51.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:51.010+0000 7f6681997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:51.019 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:51.017+0000 7f6681997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:51.210 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 89 15 22077 22062 15.4914 19.25 0.0676513 0.0645264 2024-09-22T03:15:51.250 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-22T03:15:51.251 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:15:51.251 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:15:51.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:51.409+0000 7f2557397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:51.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:51.421+0000 7f2556135640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:51.768 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:15:51.769 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:15:51.787 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-22T03:15:51.280878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1399259699,"num_objects":21346,"num_object_clones":0,"num_object_copies":64038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":21481,"num_write_kb":1367784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21426,"ondisk_log_size":21426,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":496632,"kb_used_data":256180,"kb_used_omap":0,"kb_used_meta":240384,"kb_avail":376990728,"statfs":{"total":386547056640,"available":386038505472,"internally_reserved":0,"allocated":262328320,"data_stored":4164157569,"data_compressed":197701695,"data_compressed_allocated":260222976,"data_compressed_original":4163321856,"omap_allocated":0,"internal_metadata":246153216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":279707648,"num_objects":4268,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4268,"num_write_kb":273152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002305"},"pg_stats":[{"pgid":"4.f","version":"61'1279","reported_seq":1301,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:48.256963+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:15:48.256963+0000","last_peered":"2024-09-22T03:15:48.256963+0000","last_clean":"2024-09-22T03:15:48.256963+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:15:48.256963+0000","last_undegraded":"2024-09-22T03:15:48.256963+0000","last_fullsized":"2024-09-22T03:15:48.256963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1279,"log_dups_size":0,"ondisk_log_size":1279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":1280,"num_object_clones":0,"num_object_copies":3840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1280,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"61'1368","reported_seq":1391,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:48.155176+0000","last_change":"2024-09-22T03:15:36.121590+0000","last_active":"2024-09-22T03:15:48.155176+0000","last_peered":"2024-09-22T03:15:48.155176+0000","last_clean":"2024-09-22T03:15:48.155176+0000","last_became_active":"2024-09-22T03:15:36.113187+0000","last_became_peered":"2024-09-22T03:15:36.113187+0000","last_unstale":"2024-09-22T03:15:48.155176+0000","last_undegraded":"2024-09-22T03:15:48.155176+0000","last_fullsized":"2024-09-22T03:15:48.155176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89718784,"num_objects":1369,"num_object_clones":0,"num_object_copies":4107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1369,"num_write_kb":87616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'1342","reported_seq":1364,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:49.848277+0000","last_change":"2024-09-22T03:15:36.222864+0000","last_active":"2024-09-22T03:15:49.848277+0000","last_peered":"2024-09-22T03:15:49.848277+0000","last_clean":"2024-09-22T03:15:49.848277+0000","last_became_active":"2024-09-22T03:15:36.212587+0000","last_became_peered":"2024-09-22T03:15:36.212587+0000","last_unstale":"2024-09-22T03:15:49.848277+0000","last_undegraded":"2024-09-22T03:15:49.848277+0000","last_fullsized":"2024-09-22T03:15:49.848277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88014848,"num_objects":1343,"num_object_clones":0,"num_object_copies":4029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1343,"num_write_kb":85952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"61'1328","reported_seq":1351,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:51.063818+0000","last_change":"2024-09-22T03:15:36.123944+0000","last_active":"2024-09-22T03:15:51.063818+0000","last_peered":"2024-09-22T03:15:51.063818+0000","last_clean":"2024-09-22T03:15:51.063818+0000","last_became_active":"2024-09-22T03:15:36.120054+0000","last_became_peered":"2024-09-22T03:15:36.120054+0000","last_unstale":"2024-09-22T03:15:51.063818+0000","last_undegraded":"2024-09-22T03:15:51.063818+0000","last_fullsized":"2024-09-22T03:15:51.063818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1328,"log_dups_size":0,"ondisk_log_size":1328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87097344,"num_objects":1329,"num_object_clones":0,"num_object_copies":3987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1329,"num_write_kb":85056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'1300","reported_seq":1328,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:47.140414+0000","last_change":"2024-09-22T03:15:36.315101+0000","last_active":"2024-09-22T03:15:47.140414+0000","last_peered":"2024-09-22T03:15:47.140414+0000","last_clean":"2024-09-22T03:15:47.140414+0000","last_became_active":"2024-09-22T03:15:36.164467+0000","last_became_peered":"2024-09-22T03:15:36.164467+0000","last_unstale":"2024-09-22T03:15:47.140414+0000","last_undegraded":"2024-09-22T03:15:47.140414+0000","last_fullsized":"2024-09-22T03:15:47.140414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1300,"log_dups_size":0,"ondisk_log_size":1300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85262336,"num_objects":1301,"num_object_clones":0,"num_object_copies":3903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1301,"num_write_kb":83264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'1397","reported_seq":1419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:49.732932+0000","last_change":"2024-09-22T03:15:36.164040+0000","last_active":"2024-09-22T03:15:49.732932+0000","last_peered":"2024-09-22T03:15:49.732932+0000","last_clean":"2024-09-22T03:15:49.732932+0000","last_became_active":"2024-09-22T03:15:36.150668+0000","last_became_peered":"2024-09-22T03:15:36.150668+0000","last_unstale":"2024-09-22T03:15:49.732932+0000","last_undegraded":"2024-09-22T03:15:49.732932+0000","last_fullsized":"2024-09-22T03:15:49.732932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1397,"log_dups_size":0,"ondisk_log_size":1397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91619328,"num_objects":1398,"num_object_clones":0,"num_object_copies":4194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1398,"num_write_kb":89472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"61'1322","reported_seq":1344,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:49.778658+0000","last_change":"2024-09-22T03:15:36.199911+0000","last_active":"2024-09-22T03:15:49.778658+0000","last_peered":"2024-09-22T03:15:49.778658+0000","last_clean":"2024-09-22T03:15:49.778658+0000","last_became_active":"2024-09-22T03:15:36.193383+0000","last_became_peered":"2024-09-22T03:15:36.193383+0000","last_unstale":"2024-09-22T03:15:49.778658+0000","last_undegraded":"2024-09-22T03:15:49.778658+0000","last_fullsized":"2024-09-22T03:15:49.778658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1322,"log_dups_size":0,"ondisk_log_size":1322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86704128,"num_objects":1323,"num_object_clones":0,"num_object_copies":3969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1323,"num_write_kb":84672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"61'1323","reported_seq":1345,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:49.837147+0000","last_change":"2024-09-22T03:15:36.198659+0000","last_active":"2024-09-22T03:15:49.837147+0000","last_peered":"2024-09-22T03:15:49.837147+0000","last_clean":"2024-09-22T03:15:49.837147+0000","last_became_active":"2024-09-22T03:15:36.191916+0000","last_became_peered":"2024-09-22T03:15:36.191916+0000","last_unstale":"2024-09-22T03:15:49.837147+0000","last_undegraded":"2024-09-22T03:15:49.837147+0000","last_fullsized":"2024-09-22T03:15:49.837147+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86769664,"num_objects":1324,"num_object_clones":0,"num_object_copies":3972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1324,"num_write_kb":84736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'1369","reported_seq":1391,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:49.734120+0000","last_change":"2024-09-22T03:15:36.172715+0000","last_active":"2024-09-22T03:15:49.734120+0000","last_peered":"2024-09-22T03:15:49.734120+0000","last_clean":"2024-09-22T03:15:49.734120+0000","last_became_active":"2024-09-22T03:15:36.167310+0000","last_became_peered":"2024-09-22T03:15:36.167310+0000","last_unstale":"2024-09-22T03:15:49.734120+0000","last_undegraded":"2024-09-22T03:15:49.734120+0000","last_fullsized":"2024-09-22T03:15:49.734120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1369,"log_dups_size":0,"ondisk_log_size":1369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89784320,"num_objects":1370,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1370,"num_write_kb":87680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.065433+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:15:36.065433+0000","last_peered":"2024-09-22T03:15:36.065433+0000","last_clean":"2024-09-22T03:15:36.065433+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:15:36.065433+0000","last_undegraded":"2024-09-22T03:15:36.065433+0000","last_fullsized":"2024-09-22T03:15:36.065433+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"61'1366","reported_seq":1394,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:48.219439+0000","last_change":"2024-09-22T03:15:36.205087+0000","last_active":"2024-09-22T03:15:48.219439+0000","last_peered":"2024-09-22T03:15:48.219439+0000","last_clean":"2024-09-22T03:15:48.219439+0000","last_became_active":"2024-09-22T03:15:36.122754+0000","last_became_peered":"2024-09-22T03:15:36.122754+0000","last_unstale":"2024-09-22T03:15:48.219439+0000","last_undegraded":"2024-09-22T03:15:48.219439+0000","last_fullsized":"2024-09-22T03:15:48.219439+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1366,"log_dups_size":0,"ondisk_log_size":1366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89587712,"num_objects":1367,"num_object_clones":0,"num_object_copies":4101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1367,"num_write_kb":87488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":165,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.135215+0000","last_change":"2024-09-22T03:15:36.136065+0000","last_active":"2024-09-22T03:15:36.135215+0000","last_peered":"2024-09-22T03:15:36.135215+0000","last_clean":"2024-09-22T03:14:22.124679+0000","last_became_active":"2024-09-22T03:15:36.128652+0000","last_became_peered":"2024-09-22T03:15:36.128652+0000","last_unstale":"2024-09-22T03:15:36.135215+0000","last_undegraded":"2024-09-22T03:15:36.135215+0000","last_fullsized":"2024-09-22T03:15:36.135215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"61'1293","reported_seq":1315,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:48.233719+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:15:48.233719+0000","last_peered":"2024-09-22T03:15:48.233719+0000","last_clean":"2024-09-22T03:15:48.233719+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:15:48.233719+0000","last_undegraded":"2024-09-22T03:15:48.233719+0000","last_fullsized":"2024-09-22T03:15:48.233719+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1293,"log_dups_size":0,"ondisk_log_size":1293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84803584,"num_objects":1294,"num_object_clones":0,"num_object_copies":3882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1294,"num_write_kb":82816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":169,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.123704+0000","last_change":"2024-09-22T03:15:36.124379+0000","last_active":"2024-09-22T03:15:36.123704+0000","last_peered":"2024-09-22T03:15:36.123704+0000","last_clean":"2024-09-22T03:14:22.126825+0000","last_became_active":"2024-09-22T03:15:36.116333+0000","last_became_peered":"2024-09-22T03:15:36.116333+0000","last_unstale":"2024-09-22T03:15:36.123704+0000","last_undegraded":"2024-09-22T03:15:36.123704+0000","last_fullsized":"2024-09-22T03:15:36.123704+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"61'1268","reported_seq":1290,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:47.127410+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:15:47.127410+0000","last_peered":"2024-09-22T03:15:47.127410+0000","last_clean":"2024-09-22T03:15:47.127410+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:15:47.127410+0000","last_undegraded":"2024-09-22T03:15:47.127410+0000","last_fullsized":"2024-09-22T03:15:47.127410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1268,"log_dups_size":0,"ondisk_log_size":1268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83165184,"num_objects":1269,"num_object_clones":0,"num_object_copies":3807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1269,"num_write_kb":81216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.146621+0000","last_change":"2024-09-22T03:15:36.147591+0000","last_active":"2024-09-22T03:15:36.146621+0000","last_peered":"2024-09-22T03:15:36.146621+0000","last_clean":"2024-09-22T03:14:22.130818+0000","last_became_active":"2024-09-22T03:15:36.142273+0000","last_became_peered":"2024-09-22T03:15:36.142273+0000","last_unstale":"2024-09-22T03:15:36.146621+0000","last_undegraded":"2024-09-22T03:15:36.146621+0000","last_fullsized":"2024-09-22T03:15:36.146621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":289,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.145409+0000","last_change":"2024-09-22T03:15:36.145998+0000","last_active":"2024-09-22T03:15:36.145409+0000","last_peered":"2024-09-22T03:15:36.145409+0000","last_clean":"2024-09-22T03:14:22.129493+0000","last_became_active":"2024-09-22T03:15:36.141917+0000","last_became_peered":"2024-09-22T03:15:36.141917+0000","last_unstale":"2024-09-22T03:15:36.145409+0000","last_undegraded":"2024-09-22T03:15:36.145409+0000","last_fullsized":"2024-09-22T03:15:36.145409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"61'1343","reported_seq":1365,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:49.708934+0000","last_change":"2024-09-22T03:15:36.211893+0000","last_active":"2024-09-22T03:15:49.708934+0000","last_peered":"2024-09-22T03:15:49.708934+0000","last_clean":"2024-09-22T03:15:49.708934+0000","last_became_active":"2024-09-22T03:15:36.205362+0000","last_became_peered":"2024-09-22T03:15:36.205362+0000","last_unstale":"2024-09-22T03:15:49.708934+0000","last_undegraded":"2024-09-22T03:15:49.708934+0000","last_fullsized":"2024-09-22T03:15:49.708934+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":1344,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1344,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.224997+0000","last_change":"2024-09-22T03:15:36.225509+0000","last_active":"2024-09-22T03:15:36.224997+0000","last_peered":"2024-09-22T03:15:36.224997+0000","last_clean":"2024-09-22T03:12:31.099628+0000","last_became_active":"2024-09-22T03:15:36.222016+0000","last_became_peered":"2024-09-22T03:15:36.222016+0000","last_unstale":"2024-09-22T03:15:36.224997+0000","last_undegraded":"2024-09-22T03:15:36.224997+0000","last_fullsized":"2024-09-22T03:15:36.224997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"61'1304","reported_seq":1332,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:47.199147+0000","last_change":"2024-09-22T03:15:36.285056+0000","last_active":"2024-09-22T03:15:47.199147+0000","last_peered":"2024-09-22T03:15:47.199147+0000","last_clean":"2024-09-22T03:15:47.199147+0000","last_became_active":"2024-09-22T03:15:36.150654+0000","last_became_peered":"2024-09-22T03:15:36.150654+0000","last_unstale":"2024-09-22T03:15:47.199147+0000","last_undegraded":"2024-09-22T03:15:47.199147+0000","last_fullsized":"2024-09-22T03:15:47.199147+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1304,"log_dups_size":0,"ondisk_log_size":1304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85524480,"num_objects":1305,"num_object_clones":0,"num_object_copies":3915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1305,"num_write_kb":83520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":159,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.068093+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:15:36.068093+0000","last_peered":"2024-09-22T03:15:36.068093+0000","last_clean":"2024-09-22T03:15:36.068093+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:15:36.068093+0000","last_undegraded":"2024-09-22T03:15:36.068093+0000","last_fullsized":"2024-09-22T03:15:36.068093+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"61'1326","reported_seq":1356,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:47.234221+0000","last_change":"2024-09-22T03:15:36.326798+0000","last_active":"2024-09-22T03:15:47.234221+0000","last_peered":"2024-09-22T03:15:47.234221+0000","last_clean":"2024-09-22T03:15:47.234221+0000","last_became_active":"2024-09-22T03:15:36.132909+0000","last_became_peered":"2024-09-22T03:15:36.132909+0000","last_unstale":"2024-09-22T03:15:47.234221+0000","last_undegraded":"2024-09-22T03:15:47.234221+0000","last_fullsized":"2024-09-22T03:15:47.234221+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1326,"log_dups_size":0,"ondisk_log_size":1326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86966272,"num_objects":1327,"num_object_clones":0,"num_object_copies":3981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":84928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":121,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.125115+0000","last_change":"2024-09-22T03:15:36.125988+0000","last_active":"2024-09-22T03:15:36.125115+0000","last_peered":"2024-09-22T03:15:36.125115+0000","last_clean":"2024-09-22T03:14:07.203502+0000","last_became_active":"2024-09-22T03:15:36.121933+0000","last_became_peered":"2024-09-22T03:15:36.121933+0000","last_unstale":"2024-09-22T03:15:36.125115+0000","last_undegraded":"2024-09-22T03:15:36.125115+0000","last_fullsized":"2024-09-22T03:15:36.125115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"61'1398","reported_seq":1420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:49.807477+0000","last_change":"2024-09-22T03:15:36.136827+0000","last_active":"2024-09-22T03:15:49.807477+0000","last_peered":"2024-09-22T03:15:49.807477+0000","last_clean":"2024-09-22T03:15:49.807477+0000","last_became_active":"2024-09-22T03:15:36.130890+0000","last_became_peered":"2024-09-22T03:15:36.130890+0000","last_unstale":"2024-09-22T03:15:49.807477+0000","last_undegraded":"2024-09-22T03:15:49.807477+0000","last_fullsized":"2024-09-22T03:15:49.807477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1398,"log_dups_size":0,"ondisk_log_size":1398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91684864,"num_objects":1399,"num_object_clones":0,"num_object_copies":4197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":89536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.182260+0000","last_change":"2024-09-22T03:15:36.182916+0000","last_active":"2024-09-22T03:15:36.182260+0000","last_peered":"2024-09-22T03:15:36.182260+0000","last_clean":"2024-09-22T03:14:07.210158+0000","last_became_active":"2024-09-22T03:15:36.173982+0000","last_became_peered":"2024-09-22T03:15:36.173982+0000","last_unstale":"2024-09-22T03:15:36.182260+0000","last_undegraded":"2024-09-22T03:15:36.182260+0000","last_fullsized":"2024-09-22T03:15:36.182260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":1398669312,"num_objects":21342,"num_object_clones":0,"num_object_copies":64026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21342,"num_write_kb":1365888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":263430144,"data_stored":4214882304,"data_compressed":200141835,"data_compressed_allocated":263430144,"data_compressed_original":4214882304,"omap_allocated":0,"internal_metadata":0},"log_size":21326,"ondisk_log_size":21326,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266792,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127928,"kb_used_data":71140,"kb_used_omap":0,"kb_used_meta":56768,"kb_avail":94243912,"statfs":{"total":96636764160,"available":96505765888,"internally_reserved":0,"allocated":72847360,"data_stored":1157561997,"data_compressed":54957027,"data_compressed_allocated":72335360,"data_compressed_original":1157365760,"omap_allocated":0,"internal_metadata":58130432},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673244,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":120240,"kb_used_data":53152,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94251600,"statfs":{"total":96636764160,"available":96513638400,"internally_reserved":0,"allocated":54427648,"data_stored":862453933,"data_compressed":40945846,"data_compressed_allocated":53895168,"data_compressed_original":862240768,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":60,"seq":257698037774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119936,"kb_used_data":70448,"kb_used_omap":0,"kb_used_meta":49472,"kb_avail":94251904,"statfs":{"total":96636764160,"available":96513949696,"internally_reserved":0,"allocated":72138752,"data_stored":1145831597,"data_compressed":54401935,"data_compressed_allocated":71606272,"data_compressed_original":1145618432,"omap_allocated":0,"internal_metadata":50659328},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":128528,"kb_used_data":61440,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94243312,"statfs":{"total":96636764160,"available":96505151488,"internally_reserved":0,"allocated":62914560,"data_stored":998310042,"data_compressed":47396887,"data_compressed_allocated":62386176,"data_compressed_original":998096896,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63131648,"data_stored":1010106368,"data_compressed":47964419,"data_compressed_allocated":63131648,"data_compressed_original":1010106368,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72417280,"data_stored":1158676480,"data_compressed":55019259,"data_compressed_allocated":72417280,"data_compressed_original":1158676480,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54562816,"data_stored":873005056,"data_compressed":41454250,"data_compressed_allocated":54562816,"data_compressed_original":873005056,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73318400,"data_stored":1173094400,"data_compressed":55703907,"data_compressed_allocated":73318400,"data_compressed_original":1173094400,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:15:52.210 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 90 15 22348 22333 15.5074 16.9375 0.079684 0.06444 2024-09-22T03:15:53.210 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total time run: 90.049 2024-09-22T03:15:53.210 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total writes made: 22348 2024-09-22T03:15:53.210 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Write size: 65536 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object size: 65536 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Bandwidth (MB/sec): 15.511 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Bandwidth: 10.6998 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max bandwidth (MB/sec): 27.25 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min bandwidth (MB/sec): 0 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average IOPS: 248 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev IOPS: 171.199 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max IOPS: 436 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min IOPS: 0 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average Latency(s): 0.0644528 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Latency(s): 0.758467 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max latency(s): 28.4584 2024-09-22T03:15:53.211 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min latency(s): 0.00928913 2024-09-22T03:15:53.228 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:15:53.281 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:15:53.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:53.421+0000 7fb09f6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:53.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:53.429+0000 7fb09f6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:55.649 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:15:55.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:55.806+0000 7f4efe82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:55.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:55.813+0000 7f4efe82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:56.789 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-22T03:15:56.789 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:15:56.790 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:15:56.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:56.950+0000 7fdd0ab4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:56.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:56.960+0000 7fdd0ab4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:57.320 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:15:57.321 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:15:57.338 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-22T03:15:57.282069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1420755531,"num_objects":21675,"num_object_clones":0,"num_object_copies":65025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":23166,"num_write_kb":1432169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23111,"ondisk_log_size":23111,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":508412,"kb_used_data":263800,"kb_used_omap":0,"kb_used_meta":244544,"kb_avail":376978948,"statfs":{"total":386547056640,"available":386026442752,"internally_reserved":0,"allocated":270131200,"data_stored":4288872625,"data_compressed":203625626,"data_compressed_allocated":268017664,"data_compressed_original":4288036864,"omap_allocated":0,"internal_metadata":250413056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131858456,"num_objects":2013,"num_object_clones":0,"num_object_copies":6039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3369,"num_write_kb":172161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002340"},"pg_stats":[{"pgid":"4.f","version":"61'1352","reported_seq":1374,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:53.261769+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:15:53.261769+0000","last_peered":"2024-09-22T03:15:53.261769+0000","last_clean":"2024-09-22T03:15:53.261769+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:15:53.261769+0000","last_undegraded":"2024-09-22T03:15:53.261769+0000","last_fullsized":"2024-09-22T03:15:53.261769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1352,"log_dups_size":0,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88145920,"num_objects":1345,"num_object_clones":0,"num_object_copies":4035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1353,"num_write_kb":86336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"61'1436","reported_seq":1459,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:52.144166+0000","last_change":"2024-09-22T03:15:36.121590+0000","last_active":"2024-09-22T03:15:52.144166+0000","last_peered":"2024-09-22T03:15:52.144166+0000","last_clean":"2024-09-22T03:15:52.144166+0000","last_became_active":"2024-09-22T03:15:36.113187+0000","last_became_peered":"2024-09-22T03:15:36.113187+0000","last_unstale":"2024-09-22T03:15:52.144166+0000","last_undegraded":"2024-09-22T03:15:52.144166+0000","last_fullsized":"2024-09-22T03:15:52.144166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1436,"log_dups_size":0,"ondisk_log_size":1436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94044160,"num_objects":1435,"num_object_clones":0,"num_object_copies":4305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1437,"num_write_kb":91904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'1417","reported_seq":1440,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:54.814772+0000","last_change":"2024-09-22T03:15:36.222864+0000","last_active":"2024-09-22T03:15:54.814772+0000","last_peered":"2024-09-22T03:15:54.814772+0000","last_clean":"2024-09-22T03:15:54.814772+0000","last_became_active":"2024-09-22T03:15:36.212587+0000","last_became_peered":"2024-09-22T03:15:36.212587+0000","last_unstale":"2024-09-22T03:15:54.814772+0000","last_undegraded":"2024-09-22T03:15:54.814772+0000","last_fullsized":"2024-09-22T03:15:54.814772+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1417,"log_dups_size":0,"ondisk_log_size":1417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88670232,"num_objects":1354,"num_object_clones":0,"num_object_copies":4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1418,"num_write_kb":88641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"61'1400","reported_seq":1423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:55.931748+0000","last_change":"2024-09-22T03:15:36.123944+0000","last_active":"2024-09-22T03:15:55.931748+0000","last_peered":"2024-09-22T03:15:55.931748+0000","last_clean":"2024-09-22T03:15:55.931748+0000","last_became_active":"2024-09-22T03:15:36.120054+0000","last_became_peered":"2024-09-22T03:15:36.120054+0000","last_unstale":"2024-09-22T03:15:55.931748+0000","last_undegraded":"2024-09-22T03:15:55.931748+0000","last_fullsized":"2024-09-22T03:15:55.931748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85393408,"num_objects":1303,"num_object_clones":0,"num_object_copies":3909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1401,"num_write_kb":86528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'1497","reported_seq":1525,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.150380+0000","last_change":"2024-09-22T03:15:36.315101+0000","last_active":"2024-09-22T03:15:57.150380+0000","last_peered":"2024-09-22T03:15:57.150380+0000","last_clean":"2024-09-22T03:15:57.150380+0000","last_became_active":"2024-09-22T03:15:36.164467+0000","last_became_peered":"2024-09-22T03:15:36.164467+0000","last_unstale":"2024-09-22T03:15:57.150380+0000","last_undegraded":"2024-09-22T03:15:57.150380+0000","last_fullsized":"2024-09-22T03:15:57.150380+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1497,"log_dups_size":0,"ondisk_log_size":1497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84934656,"num_objects":1296,"num_object_clones":0,"num_object_copies":3888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1498,"num_write_kb":89408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'1470","reported_seq":1492,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:54.720275+0000","last_change":"2024-09-22T03:15:36.164040+0000","last_active":"2024-09-22T03:15:54.720275+0000","last_peered":"2024-09-22T03:15:54.720275+0000","last_clean":"2024-09-22T03:15:54.720275+0000","last_became_active":"2024-09-22T03:15:36.150668+0000","last_became_peered":"2024-09-22T03:15:36.150668+0000","last_unstale":"2024-09-22T03:15:54.720275+0000","last_undegraded":"2024-09-22T03:15:54.720275+0000","last_fullsized":"2024-09-22T03:15:54.720275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92995584,"num_objects":1419,"num_object_clones":0,"num_object_copies":4257,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1471,"num_write_kb":92480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"61'1412","reported_seq":1434,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:54.788504+0000","last_change":"2024-09-22T03:15:36.199911+0000","last_active":"2024-09-22T03:15:54.788504+0000","last_peered":"2024-09-22T03:15:54.788504+0000","last_clean":"2024-09-22T03:15:54.788504+0000","last_became_active":"2024-09-22T03:15:36.193383+0000","last_became_peered":"2024-09-22T03:15:36.193383+0000","last_unstale":"2024-09-22T03:15:54.788504+0000","last_undegraded":"2024-09-22T03:15:54.788504+0000","last_fullsized":"2024-09-22T03:15:54.788504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87228416,"num_objects":1331,"num_object_clones":0,"num_object_copies":3993,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1413,"num_write_kb":87808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"61'1393","reported_seq":1415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:54.742625+0000","last_change":"2024-09-22T03:15:36.198659+0000","last_active":"2024-09-22T03:15:54.742625+0000","last_peered":"2024-09-22T03:15:54.742625+0000","last_clean":"2024-09-22T03:15:54.742625+0000","last_became_active":"2024-09-22T03:15:36.191916+0000","last_became_peered":"2024-09-22T03:15:36.191916+0000","last_unstale":"2024-09-22T03:15:54.742625+0000","last_undegraded":"2024-09-22T03:15:54.742625+0000","last_fullsized":"2024-09-22T03:15:54.742625+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1393,"log_dups_size":0,"ondisk_log_size":1393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87031808,"num_objects":1328,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1394,"num_write_kb":87104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'1456","reported_seq":1478,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:54.755320+0000","last_change":"2024-09-22T03:15:36.172715+0000","last_active":"2024-09-22T03:15:54.755320+0000","last_peered":"2024-09-22T03:15:54.755320+0000","last_clean":"2024-09-22T03:15:54.755320+0000","last_became_active":"2024-09-22T03:15:36.167310+0000","last_became_peered":"2024-09-22T03:15:36.167310+0000","last_unstale":"2024-09-22T03:15:54.755320+0000","last_undegraded":"2024-09-22T03:15:54.755320+0000","last_fullsized":"2024-09-22T03:15:54.755320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90374144,"num_objects":1379,"num_object_clones":0,"num_object_copies":4137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1457,"num_write_kb":90752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.065433+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:15:36.065433+0000","last_peered":"2024-09-22T03:15:36.065433+0000","last_clean":"2024-09-22T03:15:36.065433+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:15:36.065433+0000","last_undegraded":"2024-09-22T03:15:36.065433+0000","last_fullsized":"2024-09-22T03:15:36.065433+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"61'1429","reported_seq":1457,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:53.233234+0000","last_change":"2024-09-22T03:15:36.205087+0000","last_active":"2024-09-22T03:15:53.233234+0000","last_peered":"2024-09-22T03:15:53.233234+0000","last_clean":"2024-09-22T03:15:53.233234+0000","last_became_active":"2024-09-22T03:15:36.122754+0000","last_became_peered":"2024-09-22T03:15:36.122754+0000","last_unstale":"2024-09-22T03:15:53.233234+0000","last_undegraded":"2024-09-22T03:15:53.233234+0000","last_fullsized":"2024-09-22T03:15:53.233234+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1429,"log_dups_size":0,"ondisk_log_size":1429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93454336,"num_objects":1426,"num_object_clones":0,"num_object_copies":4278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1430,"num_write_kb":91392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":165,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.135215+0000","last_change":"2024-09-22T03:15:36.136065+0000","last_active":"2024-09-22T03:15:36.135215+0000","last_peered":"2024-09-22T03:15:36.135215+0000","last_clean":"2024-09-22T03:14:22.124679+0000","last_became_active":"2024-09-22T03:15:36.128652+0000","last_became_peered":"2024-09-22T03:15:36.128652+0000","last_unstale":"2024-09-22T03:15:36.135215+0000","last_undegraded":"2024-09-22T03:15:36.135215+0000","last_fullsized":"2024-09-22T03:15:36.135215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"61'1368","reported_seq":1390,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:52.117757+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:15:52.117757+0000","last_peered":"2024-09-22T03:15:52.117757+0000","last_clean":"2024-09-22T03:15:52.117757+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:15:52.117757+0000","last_undegraded":"2024-09-22T03:15:52.117757+0000","last_fullsized":"2024-09-22T03:15:52.117757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89587712,"num_objects":1367,"num_object_clones":0,"num_object_copies":4101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1369,"num_write_kb":87552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":169,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.123704+0000","last_change":"2024-09-22T03:15:36.124379+0000","last_active":"2024-09-22T03:15:36.123704+0000","last_peered":"2024-09-22T03:15:36.123704+0000","last_clean":"2024-09-22T03:14:22.126825+0000","last_became_active":"2024-09-22T03:15:36.116333+0000","last_became_peered":"2024-09-22T03:15:36.116333+0000","last_unstale":"2024-09-22T03:15:36.123704+0000","last_undegraded":"2024-09-22T03:15:36.123704+0000","last_fullsized":"2024-09-22T03:15:36.123704+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"61'1466","reported_seq":1488,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.183664+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:15:57.183664+0000","last_peered":"2024-09-22T03:15:57.183664+0000","last_clean":"2024-09-22T03:15:57.183664+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:15:57.183664+0000","last_undegraded":"2024-09-22T03:15:57.183664+0000","last_fullsized":"2024-09-22T03:15:57.183664+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1466,"log_dups_size":0,"ondisk_log_size":1466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84344832,"num_objects":1287,"num_object_clones":0,"num_object_copies":3861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1467,"num_write_kb":88128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.146621+0000","last_change":"2024-09-22T03:15:36.147591+0000","last_active":"2024-09-22T03:15:36.146621+0000","last_peered":"2024-09-22T03:15:36.146621+0000","last_clean":"2024-09-22T03:14:22.130818+0000","last_became_active":"2024-09-22T03:15:36.142273+0000","last_became_peered":"2024-09-22T03:15:36.142273+0000","last_unstale":"2024-09-22T03:15:36.146621+0000","last_undegraded":"2024-09-22T03:15:36.146621+0000","last_fullsized":"2024-09-22T03:15:36.146621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":289,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.145409+0000","last_change":"2024-09-22T03:15:36.145998+0000","last_active":"2024-09-22T03:15:36.145409+0000","last_peered":"2024-09-22T03:15:36.145409+0000","last_clean":"2024-09-22T03:14:22.129493+0000","last_became_active":"2024-09-22T03:15:36.141917+0000","last_became_peered":"2024-09-22T03:15:36.141917+0000","last_unstale":"2024-09-22T03:15:36.145409+0000","last_undegraded":"2024-09-22T03:15:36.145409+0000","last_fullsized":"2024-09-22T03:15:36.145409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"61'1435","reported_seq":1457,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:54.833497+0000","last_change":"2024-09-22T03:15:36.211893+0000","last_active":"2024-09-22T03:15:54.833497+0000","last_peered":"2024-09-22T03:15:54.833497+0000","last_clean":"2024-09-22T03:15:54.833497+0000","last_became_active":"2024-09-22T03:15:36.205362+0000","last_became_peered":"2024-09-22T03:15:36.205362+0000","last_unstale":"2024-09-22T03:15:54.833497+0000","last_undegraded":"2024-09-22T03:15:54.833497+0000","last_fullsized":"2024-09-22T03:15:54.833497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88866816,"num_objects":1356,"num_object_clones":0,"num_object_copies":4068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":89344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.224997+0000","last_change":"2024-09-22T03:15:36.225509+0000","last_active":"2024-09-22T03:15:36.224997+0000","last_peered":"2024-09-22T03:15:36.224997+0000","last_clean":"2024-09-22T03:12:31.099628+0000","last_became_active":"2024-09-22T03:15:36.222016+0000","last_became_peered":"2024-09-22T03:15:36.222016+0000","last_unstale":"2024-09-22T03:15:36.224997+0000","last_undegraded":"2024-09-22T03:15:36.224997+0000","last_fullsized":"2024-09-22T03:15:36.224997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"61'1481","reported_seq":1509,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.132078+0000","last_change":"2024-09-22T03:15:36.285056+0000","last_active":"2024-09-22T03:15:57.132078+0000","last_peered":"2024-09-22T03:15:57.132078+0000","last_clean":"2024-09-22T03:15:57.132078+0000","last_became_active":"2024-09-22T03:15:36.150654+0000","last_became_peered":"2024-09-22T03:15:36.150654+0000","last_unstale":"2024-09-22T03:15:57.132078+0000","last_undegraded":"2024-09-22T03:15:57.132078+0000","last_fullsized":"2024-09-22T03:15:57.132078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86638592,"num_objects":1322,"num_object_clones":0,"num_object_copies":3966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1482,"num_write_kb":89728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":159,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.068093+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:15:36.068093+0000","last_peered":"2024-09-22T03:15:36.068093+0000","last_clean":"2024-09-22T03:15:36.068093+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:15:36.068093+0000","last_undegraded":"2024-09-22T03:15:36.068093+0000","last_fullsized":"2024-09-22T03:15:36.068093+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"61'1509","reported_seq":1539,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.227926+0000","last_change":"2024-09-22T03:15:36.326798+0000","last_active":"2024-09-22T03:15:57.227926+0000","last_peered":"2024-09-22T03:15:57.227926+0000","last_clean":"2024-09-22T03:15:57.227926+0000","last_became_active":"2024-09-22T03:15:36.132909+0000","last_became_peered":"2024-09-22T03:15:36.132909+0000","last_unstale":"2024-09-22T03:15:57.227926+0000","last_undegraded":"2024-09-22T03:15:57.227926+0000","last_fullsized":"2024-09-22T03:15:57.227926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1509,"log_dups_size":0,"ondisk_log_size":1509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87031808,"num_objects":1328,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":90816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":121,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.125115+0000","last_change":"2024-09-22T03:15:36.125988+0000","last_active":"2024-09-22T03:15:36.125115+0000","last_peered":"2024-09-22T03:15:36.125115+0000","last_clean":"2024-09-22T03:14:07.203502+0000","last_became_active":"2024-09-22T03:15:36.121933+0000","last_became_peered":"2024-09-22T03:15:36.121933+0000","last_unstale":"2024-09-22T03:15:36.125115+0000","last_undegraded":"2024-09-22T03:15:36.125115+0000","last_fullsized":"2024-09-22T03:15:36.125115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"61'1490","reported_seq":1512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:54.872152+0000","last_change":"2024-09-22T03:15:36.136827+0000","last_active":"2024-09-22T03:15:54.872152+0000","last_peered":"2024-09-22T03:15:54.872152+0000","last_clean":"2024-09-22T03:15:54.872152+0000","last_became_active":"2024-09-22T03:15:36.130890+0000","last_became_peered":"2024-09-22T03:15:36.130890+0000","last_unstale":"2024-09-22T03:15:54.872152+0000","last_undegraded":"2024-09-22T03:15:54.872152+0000","last_fullsized":"2024-09-22T03:15:54.872152+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1490,"log_dups_size":0,"ondisk_log_size":1490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91422720,"num_objects":1395,"num_object_clones":0,"num_object_copies":4185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1491,"num_write_kb":92352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.182260+0000","last_change":"2024-09-22T03:15:36.182916+0000","last_active":"2024-09-22T03:15:36.182260+0000","last_peered":"2024-09-22T03:15:36.182260+0000","last_clean":"2024-09-22T03:14:07.210158+0000","last_became_active":"2024-09-22T03:15:36.173982+0000","last_became_peered":"2024-09-22T03:15:36.173982+0000","last_unstale":"2024-09-22T03:15:36.182260+0000","last_undegraded":"2024-09-22T03:15:36.182260+0000","last_fullsized":"2024-09-22T03:15:36.182260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":1420165144,"num_objects":21671,"num_object_clones":0,"num_object_copies":65013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":23027,"num_write_kb":1430273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":264982528,"data_stored":4239523912,"data_compressed":201314284,"data_compressed_allocated":264970240,"data_compressed_original":4239523840,"omap_allocated":0,"internal_metadata":0},"log_size":23011,"ondisk_log_size":23011,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131856,"kb_used_data":70912,"kb_used_omap":0,"kb_used_meta":60928,"kb_avail":94239984,"statfs":{"total":96636764160,"available":96501743616,"internally_reserved":0,"allocated":72613888,"data_stored":1153826445,"data_compressed":54780280,"data_compressed_allocated":72101888,"data_compressed_original":1153630208,"omap_allocated":0,"internal_metadata":62390272},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127996,"kb_used_data":56748,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94243844,"statfs":{"total":96636764160,"available":96505696256,"internally_reserved":0,"allocated":58109952,"data_stored":921370797,"data_compressed":43743533,"data_compressed_allocated":57577472,"data_compressed_original":921157632,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":60,"seq":257698037779,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":109640,"kb_used_data":72628,"kb_used_omap":0,"kb_used_meta":36992,"kb_avail":94262200,"statfs":{"total":96636764160,"available":96524492800,"internally_reserved":0,"allocated":74371072,"data_stored":1181483205,"data_compressed":56095186,"data_compressed_allocated":73834496,"data_compressed_original":1181270016,"omap_allocated":0,"internal_metadata":37879808},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705958,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":138920,"kb_used_data":63512,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94232920,"statfs":{"total":96636764160,"available":96494510080,"internally_reserved":0,"allocated":65036288,"data_stored":1032192178,"data_compressed":49006627,"data_compressed_allocated":64503808,"data_compressed_original":1031979008,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63696896,"data_stored":1019084824,"data_compressed":48391599,"data_compressed_allocated":63692800,"data_compressed_original":1019084800,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72785920,"data_stored":1164509208,"data_compressed":55296797,"data_compressed_allocated":72781824,"data_compressed_original":1164509184,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57475072,"data_stored":919535640,"data_compressed":43663810,"data_compressed_allocated":57470976,"data_compressed_original":919535616,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71024640,"data_stored":1136394240,"data_compressed":53962078,"data_compressed_allocated":71024640,"data_compressed_original":1136394240,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:15:58.074 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:15:58.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:58.226+0000 7fc81a597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:15:58.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:15:58.233+0000 7fc81a597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:00.475 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:16:00.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:00.633+0000 7ff115717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:00.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:00.642+0000 7ff115717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:02.340 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-22T03:16:02.340 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:16:02.341 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:16:02.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:02.484+0000 7f75a8a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:02.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:02.496+0000 7f75a8a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:02.783 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:16:02.784 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:16:02.800 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-22T03:16:01.282763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1334968907,"num_objects":20366,"num_object_clones":0,"num_object_copies":61098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":24475,"num_write_kb":1432169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24420,"ondisk_log_size":24420,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":489388,"kb_used_data":248936,"kb_used_omap":0,"kb_used_meta":240384,"kb_avail":376997972,"statfs":{"total":386547056640,"available":386045923328,"internally_reserved":0,"allocated":254910464,"data_stored":4045275337,"data_compressed":192059854,"data_compressed_allocated":252792832,"data_compressed_original":4044439552,"omap_allocated":0,"internal_metadata":246153216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12517352,"num_objects":-190,"num_object_clones":0,"num_object_copies":-570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3784,"num_write_kb":114945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002238"},"pg_stats":[{"pgid":"4.f","version":"61'1450","reported_seq":1472,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:58.277828+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:15:58.277828+0000","last_peered":"2024-09-22T03:15:58.277828+0000","last_clean":"2024-09-22T03:15:58.277828+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:15:58.277828+0000","last_undegraded":"2024-09-22T03:15:58.277828+0000","last_fullsized":"2024-09-22T03:15:58.277828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1450,"log_dups_size":0,"ondisk_log_size":1450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81723392,"num_objects":1247,"num_object_clones":0,"num_object_copies":3741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1451,"num_write_kb":86336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"61'1538","reported_seq":1561,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:58.225198+0000","last_change":"2024-09-22T03:15:36.121590+0000","last_active":"2024-09-22T03:15:58.225198+0000","last_peered":"2024-09-22T03:15:58.225198+0000","last_clean":"2024-09-22T03:15:58.225198+0000","last_became_active":"2024-09-22T03:15:36.113187+0000","last_became_peered":"2024-09-22T03:15:36.113187+0000","last_unstale":"2024-09-22T03:15:58.225198+0000","last_undegraded":"2024-09-22T03:15:58.225198+0000","last_fullsized":"2024-09-22T03:15:58.225198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1538,"log_dups_size":0,"ondisk_log_size":1538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87359488,"num_objects":1333,"num_object_clones":0,"num_object_copies":3999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1539,"num_write_kb":91904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'1537","reported_seq":1560,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:59.802728+0000","last_change":"2024-09-22T03:15:36.222864+0000","last_active":"2024-09-22T03:15:59.802728+0000","last_peered":"2024-09-22T03:15:59.802728+0000","last_clean":"2024-09-22T03:15:59.802728+0000","last_became_active":"2024-09-22T03:15:36.212587+0000","last_became_peered":"2024-09-22T03:15:36.212587+0000","last_unstale":"2024-09-22T03:15:59.802728+0000","last_undegraded":"2024-09-22T03:15:59.802728+0000","last_fullsized":"2024-09-22T03:15:59.802728+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1537,"log_dups_size":0,"ondisk_log_size":1537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80805912,"num_objects":1234,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1538,"num_write_kb":88641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"61'1521","reported_seq":1544,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:16:00.960931+0000","last_change":"2024-09-22T03:15:36.123944+0000","last_active":"2024-09-22T03:16:00.960931+0000","last_peered":"2024-09-22T03:16:00.960931+0000","last_clean":"2024-09-22T03:16:00.960931+0000","last_became_active":"2024-09-22T03:15:36.120054+0000","last_became_peered":"2024-09-22T03:15:36.120054+0000","last_unstale":"2024-09-22T03:16:00.960931+0000","last_undegraded":"2024-09-22T03:16:00.960931+0000","last_fullsized":"2024-09-22T03:16:00.960931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1521,"log_dups_size":0,"ondisk_log_size":1521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77463552,"num_objects":1182,"num_object_clones":0,"num_object_copies":3546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1522,"num_write_kb":86528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'1497","reported_seq":1525,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.150380+0000","last_change":"2024-09-22T03:15:36.315101+0000","last_active":"2024-09-22T03:15:57.150380+0000","last_peered":"2024-09-22T03:15:57.150380+0000","last_clean":"2024-09-22T03:15:57.150380+0000","last_became_active":"2024-09-22T03:15:36.164467+0000","last_became_peered":"2024-09-22T03:15:36.164467+0000","last_unstale":"2024-09-22T03:15:57.150380+0000","last_undegraded":"2024-09-22T03:15:57.150380+0000","last_fullsized":"2024-09-22T03:15:57.150380+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1497,"log_dups_size":0,"ondisk_log_size":1497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84934656,"num_objects":1296,"num_object_clones":0,"num_object_copies":3888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1498,"num_write_kb":89408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'1571","reported_seq":1593,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:59.717056+0000","last_change":"2024-09-22T03:15:36.164040+0000","last_active":"2024-09-22T03:15:59.717056+0000","last_peered":"2024-09-22T03:15:59.717056+0000","last_clean":"2024-09-22T03:15:59.717056+0000","last_became_active":"2024-09-22T03:15:36.150668+0000","last_became_peered":"2024-09-22T03:15:36.150668+0000","last_unstale":"2024-09-22T03:15:59.717056+0000","last_undegraded":"2024-09-22T03:15:59.717056+0000","last_fullsized":"2024-09-22T03:15:59.717056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1571,"log_dups_size":0,"ondisk_log_size":1571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86376448,"num_objects":1318,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1572,"num_write_kb":92480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"61'1529","reported_seq":1551,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:59.817054+0000","last_change":"2024-09-22T03:15:36.199911+0000","last_active":"2024-09-22T03:15:59.817054+0000","last_peered":"2024-09-22T03:15:59.817054+0000","last_clean":"2024-09-22T03:15:59.817054+0000","last_became_active":"2024-09-22T03:15:36.193383+0000","last_became_peered":"2024-09-22T03:15:36.193383+0000","last_unstale":"2024-09-22T03:15:59.817054+0000","last_undegraded":"2024-09-22T03:15:59.817054+0000","last_fullsized":"2024-09-22T03:15:59.817054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1529,"log_dups_size":0,"ondisk_log_size":1529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79560704,"num_objects":1214,"num_object_clones":0,"num_object_copies":3642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1530,"num_write_kb":87808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"61'1501","reported_seq":1523,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:59.873158+0000","last_change":"2024-09-22T03:15:36.198659+0000","last_active":"2024-09-22T03:15:59.873158+0000","last_peered":"2024-09-22T03:15:59.873158+0000","last_clean":"2024-09-22T03:15:59.873158+0000","last_became_active":"2024-09-22T03:15:36.191916+0000","last_became_peered":"2024-09-22T03:15:36.191916+0000","last_unstale":"2024-09-22T03:15:59.873158+0000","last_undegraded":"2024-09-22T03:15:59.873158+0000","last_fullsized":"2024-09-22T03:15:59.873158+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1501,"log_dups_size":0,"ondisk_log_size":1501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79953920,"num_objects":1220,"num_object_clones":0,"num_object_copies":3660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1502,"num_write_kb":87104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'1563","reported_seq":1585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:59.797524+0000","last_change":"2024-09-22T03:15:36.172715+0000","last_active":"2024-09-22T03:15:59.797524+0000","last_peered":"2024-09-22T03:15:59.797524+0000","last_clean":"2024-09-22T03:15:59.797524+0000","last_became_active":"2024-09-22T03:15:36.167310+0000","last_became_peered":"2024-09-22T03:15:36.167310+0000","last_unstale":"2024-09-22T03:15:59.797524+0000","last_undegraded":"2024-09-22T03:15:59.797524+0000","last_fullsized":"2024-09-22T03:15:59.797524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1563,"log_dups_size":0,"ondisk_log_size":1563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83361792,"num_objects":1272,"num_object_clones":0,"num_object_copies":3816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1564,"num_write_kb":90752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":40,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.065433+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:15:36.065433+0000","last_peered":"2024-09-22T03:15:36.065433+0000","last_clean":"2024-09-22T03:15:36.065433+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:15:36.065433+0000","last_undegraded":"2024-09-22T03:15:36.065433+0000","last_fullsized":"2024-09-22T03:15:36.065433+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"61'1540","reported_seq":1568,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:58.211008+0000","last_change":"2024-09-22T03:15:36.205087+0000","last_active":"2024-09-22T03:15:58.211008+0000","last_peered":"2024-09-22T03:15:58.211008+0000","last_clean":"2024-09-22T03:15:58.211008+0000","last_became_active":"2024-09-22T03:15:36.122754+0000","last_became_peered":"2024-09-22T03:15:36.122754+0000","last_unstale":"2024-09-22T03:15:58.211008+0000","last_undegraded":"2024-09-22T03:15:58.211008+0000","last_fullsized":"2024-09-22T03:15:58.211008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1540,"log_dups_size":0,"ondisk_log_size":1540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86179840,"num_objects":1315,"num_object_clones":0,"num_object_copies":3945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1541,"num_write_kb":91392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":165,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.135215+0000","last_change":"2024-09-22T03:15:36.136065+0000","last_active":"2024-09-22T03:15:36.135215+0000","last_peered":"2024-09-22T03:15:36.135215+0000","last_clean":"2024-09-22T03:14:22.124679+0000","last_became_active":"2024-09-22T03:15:36.128652+0000","last_became_peered":"2024-09-22T03:15:36.128652+0000","last_unstale":"2024-09-22T03:15:36.135215+0000","last_undegraded":"2024-09-22T03:15:36.135215+0000","last_fullsized":"2024-09-22T03:15:36.135215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"61'1471","reported_seq":1493,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:58.160944+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:15:58.160944+0000","last_peered":"2024-09-22T03:15:58.160944+0000","last_clean":"2024-09-22T03:15:58.160944+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:15:58.160944+0000","last_undegraded":"2024-09-22T03:15:58.160944+0000","last_fullsized":"2024-09-22T03:15:58.160944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1471,"log_dups_size":0,"ondisk_log_size":1471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82837504,"num_objects":1264,"num_object_clones":0,"num_object_copies":3792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1472,"num_write_kb":87552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":169,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.123704+0000","last_change":"2024-09-22T03:15:36.124379+0000","last_active":"2024-09-22T03:15:36.123704+0000","last_peered":"2024-09-22T03:15:36.123704+0000","last_clean":"2024-09-22T03:14:22.126825+0000","last_became_active":"2024-09-22T03:15:36.116333+0000","last_became_peered":"2024-09-22T03:15:36.116333+0000","last_unstale":"2024-09-22T03:15:36.123704+0000","last_undegraded":"2024-09-22T03:15:36.123704+0000","last_fullsized":"2024-09-22T03:15:36.123704+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"61'1466","reported_seq":1488,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.183664+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:15:57.183664+0000","last_peered":"2024-09-22T03:15:57.183664+0000","last_clean":"2024-09-22T03:15:57.183664+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:15:57.183664+0000","last_undegraded":"2024-09-22T03:15:57.183664+0000","last_fullsized":"2024-09-22T03:15:57.183664+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1466,"log_dups_size":0,"ondisk_log_size":1466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84344832,"num_objects":1287,"num_object_clones":0,"num_object_copies":3861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1467,"num_write_kb":88128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.146621+0000","last_change":"2024-09-22T03:15:36.147591+0000","last_active":"2024-09-22T03:15:36.146621+0000","last_peered":"2024-09-22T03:15:36.146621+0000","last_clean":"2024-09-22T03:14:22.130818+0000","last_became_active":"2024-09-22T03:15:36.142273+0000","last_became_peered":"2024-09-22T03:15:36.142273+0000","last_unstale":"2024-09-22T03:15:36.146621+0000","last_undegraded":"2024-09-22T03:15:36.146621+0000","last_fullsized":"2024-09-22T03:15:36.146621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":289,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.145409+0000","last_change":"2024-09-22T03:15:36.145998+0000","last_active":"2024-09-22T03:15:36.145409+0000","last_peered":"2024-09-22T03:15:36.145409+0000","last_clean":"2024-09-22T03:14:22.129493+0000","last_became_active":"2024-09-22T03:15:36.141917+0000","last_became_peered":"2024-09-22T03:15:36.141917+0000","last_unstale":"2024-09-22T03:15:36.145409+0000","last_undegraded":"2024-09-22T03:15:36.145409+0000","last_fullsized":"2024-09-22T03:15:36.145409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"61'1537","reported_seq":1559,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:59.756350+0000","last_change":"2024-09-22T03:15:36.211893+0000","last_active":"2024-09-22T03:15:59.756350+0000","last_peered":"2024-09-22T03:15:59.756350+0000","last_clean":"2024-09-22T03:15:59.756350+0000","last_became_active":"2024-09-22T03:15:36.205362+0000","last_became_peered":"2024-09-22T03:15:36.205362+0000","last_unstale":"2024-09-22T03:15:59.756350+0000","last_undegraded":"2024-09-22T03:15:59.756350+0000","last_fullsized":"2024-09-22T03:15:59.756350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1537,"log_dups_size":0,"ondisk_log_size":1537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82182144,"num_objects":1254,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1538,"num_write_kb":89344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.224997+0000","last_change":"2024-09-22T03:15:36.225509+0000","last_active":"2024-09-22T03:15:36.224997+0000","last_peered":"2024-09-22T03:15:36.224997+0000","last_clean":"2024-09-22T03:12:31.099628+0000","last_became_active":"2024-09-22T03:15:36.222016+0000","last_became_peered":"2024-09-22T03:15:36.222016+0000","last_unstale":"2024-09-22T03:15:36.224997+0000","last_undegraded":"2024-09-22T03:15:36.224997+0000","last_fullsized":"2024-09-22T03:15:36.224997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"61'1481","reported_seq":1509,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.132078+0000","last_change":"2024-09-22T03:15:36.285056+0000","last_active":"2024-09-22T03:15:57.132078+0000","last_peered":"2024-09-22T03:15:57.132078+0000","last_clean":"2024-09-22T03:15:57.132078+0000","last_became_active":"2024-09-22T03:15:36.150654+0000","last_became_peered":"2024-09-22T03:15:36.150654+0000","last_unstale":"2024-09-22T03:15:57.132078+0000","last_undegraded":"2024-09-22T03:15:57.132078+0000","last_fullsized":"2024-09-22T03:15:57.132078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86638592,"num_objects":1322,"num_object_clones":0,"num_object_copies":3966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1482,"num_write_kb":89728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":159,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.068093+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:15:36.068093+0000","last_peered":"2024-09-22T03:15:36.068093+0000","last_clean":"2024-09-22T03:15:36.068093+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:15:36.068093+0000","last_undegraded":"2024-09-22T03:15:36.068093+0000","last_fullsized":"2024-09-22T03:15:36.068093+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"61'1509","reported_seq":1539,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:57.227926+0000","last_change":"2024-09-22T03:15:36.326798+0000","last_active":"2024-09-22T03:15:57.227926+0000","last_peered":"2024-09-22T03:15:57.227926+0000","last_clean":"2024-09-22T03:15:57.227926+0000","last_became_active":"2024-09-22T03:15:36.132909+0000","last_became_peered":"2024-09-22T03:15:36.132909+0000","last_unstale":"2024-09-22T03:15:57.227926+0000","last_undegraded":"2024-09-22T03:15:57.227926+0000","last_fullsized":"2024-09-22T03:15:57.227926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1509,"log_dups_size":0,"ondisk_log_size":1509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87031808,"num_objects":1328,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":90816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":121,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.125115+0000","last_change":"2024-09-22T03:15:36.125988+0000","last_active":"2024-09-22T03:15:36.125115+0000","last_peered":"2024-09-22T03:15:36.125115+0000","last_clean":"2024-09-22T03:14:07.203502+0000","last_became_active":"2024-09-22T03:15:36.121933+0000","last_became_peered":"2024-09-22T03:15:36.121933+0000","last_unstale":"2024-09-22T03:15:36.125115+0000","last_undegraded":"2024-09-22T03:15:36.125115+0000","last_fullsized":"2024-09-22T03:15:36.125115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"61'1609","reported_seq":1631,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:59.862845+0000","last_change":"2024-09-22T03:15:36.136827+0000","last_active":"2024-09-22T03:15:59.862845+0000","last_peered":"2024-09-22T03:15:59.862845+0000","last_clean":"2024-09-22T03:15:59.862845+0000","last_became_active":"2024-09-22T03:15:36.130890+0000","last_became_peered":"2024-09-22T03:15:36.130890+0000","last_unstale":"2024-09-22T03:15:59.862845+0000","last_undegraded":"2024-09-22T03:15:59.862845+0000","last_fullsized":"2024-09-22T03:15:59.862845+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1609,"log_dups_size":0,"ondisk_log_size":1609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83623936,"num_objects":1276,"num_object_clones":0,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1610,"num_write_kb":92352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-22T03:15:36.182260+0000","last_change":"2024-09-22T03:15:36.182916+0000","last_active":"2024-09-22T03:15:36.182260+0000","last_peered":"2024-09-22T03:15:36.182260+0000","last_clean":"2024-09-22T03:14:07.210158+0000","last_became_active":"2024-09-22T03:15:36.173982+0000","last_became_peered":"2024-09-22T03:15:36.173982+0000","last_unstale":"2024-09-22T03:15:36.182260+0000","last_undegraded":"2024-09-22T03:15:36.182260+0000","last_fullsized":"2024-09-22T03:15:36.182260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":1334378520,"num_objects":20362,"num_object_clones":0,"num_object_copies":61086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":24336,"num_write_kb":1430273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249200640,"data_stored":3987013704,"data_compressed":189324738,"data_compressed_allocated":249188352,"data_compressed_original":3987013632,"omap_allocated":0,"internal_metadata":0},"log_size":24320,"ondisk_log_size":24320,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266802,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113872,"kb_used_data":65404,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94257968,"statfs":{"total":96636764160,"available":96520159232,"internally_reserved":0,"allocated":66973696,"data_stored":1063583373,"data_compressed":50495310,"data_compressed_allocated":66461696,"data_compressed_original":1063387136,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":128388,"kb_used_data":52980,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94243452,"statfs":{"total":96636764160,"available":96505294848,"internally_reserved":0,"allocated":54251520,"data_stored":859570373,"data_compressed":40809618,"data_compressed_allocated":53714944,"data_compressed_original":859357184,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":60,"seq":257698037784,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108208,"kb_used_data":67040,"kb_used_omap":0,"kb_used_meta":41152,"kb_avail":94263632,"statfs":{"total":96636764160,"available":96525959168,"internally_reserved":0,"allocated":68648960,"data_stored":1089929413,"data_compressed":51748299,"data_compressed_allocated":68112384,"data_compressed_original":1089716224,"omap_allocated":0,"internal_metadata":42139648},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705958,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":138920,"kb_used_data":63512,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94232920,"statfs":{"total":96636764160,"available":96494510080,"internally_reserved":0,"allocated":65036288,"data_stored":1032192178,"data_compressed":49006627,"data_compressed_allocated":64503808,"data_compressed_original":1031979008,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63696896,"data_stored":1019084824,"data_compressed":48391599,"data_compressed_allocated":63692800,"data_compressed_original":1019084800,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":1073807384,"data_compressed":50990119,"data_compressed_allocated":67112960,"data_compressed_original":1073807360,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52957184,"data_stored":847249432,"data_compressed":40231934,"data_compressed_allocated":52953088,"data_compressed_original":847249408,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65429504,"data_stored":1046872064,"data_compressed":49711086,"data_compressed_allocated":65429504,"data_compressed_original":1046872064,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:16:02.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:16:02.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:02.972+0000 7ffa35757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:02.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:02.981+0000 7ffa35757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:05.231 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:16:05.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:05.375+0000 7f1d184d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:05.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:05.381+0000 7f1d184d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:07.668 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:16:07.801 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-22T03:16:07.802 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:16:07.802 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 3, 1] 2024-09-22T03:16:07.802 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:16:07.802 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:16:07.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:07.811+0000 7f7bfb62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:07.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:07.816+0000 7f7bfb62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:10.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:16:13.904 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:16:15.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:16:18.904 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-22T03:16:18.905 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:16:18.905 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:16:18.905 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:16:19.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:19.019+0000 7ff9e50ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:19.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:19.026+0000 7ff9e50ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:16:19.041 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:16:19.042 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:16:19.042 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:16:19.042 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:16:19.042 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:16:19.044 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:16:19.045 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:16:19.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:16:19.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:19.144+0000 7f57fdf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:19.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:19.151+0000 7f57fdf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:36.063 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:16:36.063 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:16:36.067 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:16:36.068 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:16:36.068 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:16:36.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:16:36.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:16:36.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:36.184+0000 7ffa44663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:36.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:36.192+0000 7ffa44663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:16:36.208 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:16:36.211 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:16:36.211 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:16:36.211 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:16:36.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:36.349+0000 7f3d0ad17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:36.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:36.356+0000 7f3d0ad17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:16:36.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:16:36.372 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:16:36.372 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:16:36.374 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:16:36.374 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:16:38.172 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:16:38.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:38.278+0000 7f047f90e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:38.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:38.284+0000 7f047f90e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:38.311 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:16:38.420 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:16:38.420 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:16:40.422 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:16:40.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:40.534+0000 7fd7d8197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:40.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:40.540+0000 7fd7d8197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:40.562 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:16:40.641 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:16:40.642 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:16:41.375 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-22T03:16:41.375 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:16:41.375 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 2, 3] 2024-09-22T03:16:41.376 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T03:16:41.376 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:16:42.643 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:16:42.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:42.754+0000 7f1bfa8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:42.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:16:42.759+0000 7f1bfa8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:16:43.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-22T03:16:47.477 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T03:16:48.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-22T03:16:52.477 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1, 0] live_osds: [2, 3] 2024-09-22T03:16:52.478 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:16:52.478 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:16:52.478 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:16:52.478 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:16:52.481 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:16:52.481 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:16:52.590 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:16:52.593 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:16:52.594 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:16:57.595 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:16:57.691 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:16:57.693 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:16:57.693 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:17:02.694 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:17:02.794 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:17:02.794 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [] 2024-09-22T03:17:02.794 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:17:07.803 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 3, 1] 2024-09-22T03:17:07.804 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:17:07.804 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [2, 3, 1] 2024-09-22T03:17:07.804 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T03:17:07.804 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:17:10.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T03:17:13.904 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T03:17:15.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T03:17:18.905 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 3] live_osds: [2, 1] 2024-09-22T03:17:18.905 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:17:18.906 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:17:19.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:19.017+0000 7f8a45463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:19.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:19.024+0000 7f8a45463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:19.201 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:17:19.202 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:17:19.214 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-22T03:17:17.301220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1089929803,"num_objects":16627,"num_object_clones":0,"num_object_copies":49881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":28214,"num_write_kb":1432169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28171,"ondisk_log_size":28171,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":441056,"kb_used_data":199456,"kb_used_omap":0,"kb_used_meta":241536,"kb_avail":377046304,"statfs":{"total":386547056640,"available":386095415296,"internally_reserved":0,"allocated":204242944,"data_stored":3231915786,"data_compressed":153433710,"data_compressed_allocated":201953280,"data_compressed_original":3231006720,"omap_allocated":0,"internal_metadata":247332864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002089"},"pg_stats":[{"pgid":"4.f","version":"63'1753","reported_seq":1798,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-22T03:17:10.047777+0000","last_change":"2024-09-22T03:14:21.219315+0000","last_active":"2024-09-22T03:17:10.047777+0000","last_peered":"2024-09-22T03:17:10.047777+0000","last_clean":"2024-09-22T03:17:10.047777+0000","last_became_active":"2024-09-22T03:14:21.216272+0000","last_became_peered":"2024-09-22T03:14:21.216272+0000","last_unstale":"2024-09-22T03:17:10.047777+0000","last_undegraded":"2024-09-22T03:17:10.047777+0000","last_fullsized":"2024-09-22T03:17:10.047777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1753,"log_dups_size":0,"ondisk_log_size":1753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61931520,"num_objects":945,"num_object_clones":0,"num_object_copies":2835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1753,"num_write_kb":86336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"63'1861","reported_seq":2124,"reported_epoch":69,"state":"activating","last_fresh":"2024-09-22T03:17:10.042094+0000","last_change":"2024-09-22T03:17:08.385797+0000","last_active":"2024-09-22T03:17:07.391529+0000","last_peered":"2024-09-22T03:17:06.562515+0000","last_clean":"2024-09-22T03:16:07.669976+0000","last_became_active":"2024-09-22T03:17:05.476057+0000","last_became_peered":"2024-09-22T03:17:05.476057+0000","last_unstale":"2024-09-22T03:17:10.042094+0000","last_undegraded":"2024-09-22T03:17:10.042094+0000","last_fullsized":"2024-09-22T03:17:10.042094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1861,"log_dups_size":0,"ondisk_log_size":1861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66256896,"num_objects":1011,"num_object_clones":0,"num_object_copies":3033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":91904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":["2","1","3"],"object_location_counts":[{"shards":"1,2,3","objects":1011}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'1714","reported_seq":1743,"reported_epoch":69,"state":"peering","last_fresh":"2024-09-22T03:17:08.392298+0000","last_change":"2024-09-22T03:17:03.387806+0000","last_active":"2024-09-22T03:17:03.387806+0000","last_peered":"2024-09-22T03:16:07.733185+0000","last_clean":"2024-09-22T03:16:07.733185+0000","last_became_active":"2024-09-22T03:15:36.212587+0000","last_became_peered":"2024-09-22T03:15:36.212587+0000","last_unstale":"2024-09-22T03:17:08.392298+0000","last_undegraded":"2024-09-22T03:17:08.392298+0000","last_fullsized":"2024-09-22T03:17:08.392298+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1714,"log_dups_size":0,"ondisk_log_size":1714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69271576,"num_objects":1058,"num_object_clones":0,"num_object_copies":3174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1714,"num_write_kb":88641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'1747","reported_seq":1779,"reported_epoch":66,"state":"peering","last_fresh":"2024-09-22T03:17:04.373614+0000","last_change":"2024-09-22T03:17:03.366332+0000","last_active":"2024-09-22T03:17:03.366332+0000","last_peered":"2024-09-22T03:16:41.395673+0000","last_clean":"2024-09-22T03:16:07.774677+0000","last_became_active":"2024-09-22T03:16:37.110536+0000","last_became_peered":"2024-09-22T03:16:37.110536+0000","last_unstale":"2024-09-22T03:17:04.373614+0000","last_undegraded":"2024-09-22T03:17:04.373614+0000","last_fullsized":"2024-09-22T03:17:04.373614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1747,"log_dups_size":0,"ondisk_log_size":1747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62717952,"num_objects":957,"num_object_clones":0,"num_object_copies":2871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1747,"num_write_kb":86528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'1723","reported_seq":1758,"reported_epoch":63,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:16:37.128272+0000","last_change":"2024-09-22T03:16:37.120722+0000","last_active":"2024-09-22T03:16:37.128272+0000","last_peered":"2024-09-22T03:16:37.128272+0000","last_clean":"2024-09-22T03:16:07.799953+0000","last_became_active":"2024-09-22T03:16:37.120722+0000","last_became_peered":"2024-09-22T03:16:37.120722+0000","last_unstale":"2024-09-22T03:16:37.128272+0000","last_undegraded":"2024-09-22T03:16:37.080346+0000","last_fullsized":"2024-09-22T03:16:37.079616+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1723,"log_dups_size":0,"ondisk_log_size":1723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70123520,"num_objects":1070,"num_object_clones":0,"num_object_copies":3210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1724,"num_write_kb":89408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'1856","reported_seq":1894,"reported_epoch":69,"state":"activating","last_fresh":"2024-09-22T03:17:10.046242+0000","last_change":"2024-09-22T03:17:08.390615+0000","last_active":"2024-09-22T03:17:08.389582+0000","last_peered":"2024-09-22T03:17:06.572766+0000","last_clean":"2024-09-22T03:16:07.663835+0000","last_became_active":"2024-09-22T03:17:05.479119+0000","last_became_peered":"2024-09-22T03:17:05.479119+0000","last_unstale":"2024-09-22T03:17:10.046242+0000","last_undegraded":"2024-09-22T03:17:10.046242+0000","last_fullsized":"2024-09-22T03:17:10.046242+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1856,"log_dups_size":0,"ondisk_log_size":1856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67764224,"num_objects":1034,"num_object_clones":0,"num_object_copies":3102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1856,"num_write_kb":92480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":1034}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"61'1694","reported_seq":1722,"reported_epoch":69,"state":"peering","last_fresh":"2024-09-22T03:17:08.385567+0000","last_change":"2024-09-22T03:17:03.376805+0000","last_active":"2024-09-22T03:17:03.376805+0000","last_peered":"2024-09-22T03:16:07.756885+0000","last_clean":"2024-09-22T03:16:07.756885+0000","last_became_active":"2024-09-22T03:15:36.193383+0000","last_became_peered":"2024-09-22T03:15:36.193383+0000","last_unstale":"2024-09-22T03:17:08.385567+0000","last_undegraded":"2024-09-22T03:17:08.385567+0000","last_fullsized":"2024-09-22T03:17:08.385567+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1694,"log_dups_size":0,"ondisk_log_size":1694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68812800,"num_objects":1050,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1694,"num_write_kb":87808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"61'1665","reported_seq":1693,"reported_epoch":69,"state":"peering","last_fresh":"2024-09-22T03:17:08.388420+0000","last_change":"2024-09-22T03:17:03.395199+0000","last_active":"2024-09-22T03:17:03.395199+0000","last_peered":"2024-09-22T03:16:07.676059+0000","last_clean":"2024-09-22T03:16:07.676059+0000","last_became_active":"2024-09-22T03:15:36.191916+0000","last_became_peered":"2024-09-22T03:15:36.191916+0000","last_unstale":"2024-09-22T03:17:08.388420+0000","last_undegraded":"2024-09-22T03:17:08.388420+0000","last_fullsized":"2024-09-22T03:17:08.388420+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1665,"log_dups_size":0,"ondisk_log_size":1665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69271552,"num_objects":1057,"num_object_clones":0,"num_object_copies":3171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1665,"num_write_kb":87104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'1713","reported_seq":1741,"reported_epoch":69,"state":"peering","last_fresh":"2024-09-22T03:17:08.391988+0000","last_change":"2024-09-22T03:17:03.392122+0000","last_active":"2024-09-22T03:17:03.392122+0000","last_peered":"2024-09-22T03:16:07.777614+0000","last_clean":"2024-09-22T03:16:07.777614+0000","last_became_active":"2024-09-22T03:15:36.167310+0000","last_became_peered":"2024-09-22T03:15:36.167310+0000","last_unstale":"2024-09-22T03:17:08.391988+0000","last_undegraded":"2024-09-22T03:17:08.391988+0000","last_fullsized":"2024-09-22T03:17:08.391988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1713,"log_dups_size":0,"ondisk_log_size":1713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73596928,"num_objects":1123,"num_object_clones":0,"num_object_copies":3369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1713,"num_write_kb":90752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":55,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-22T03:17:05.381994+0000","last_change":"2024-09-22T03:14:19.178082+0000","last_active":"2024-09-22T03:17:05.381994+0000","last_peered":"2024-09-22T03:17:05.381994+0000","last_clean":"2024-09-22T03:17:05.381994+0000","last_became_active":"2024-09-22T03:14:07.165423+0000","last_became_peered":"2024-09-22T03:14:07.165423+0000","last_unstale":"2024-09-22T03:17:05.381994+0000","last_undegraded":"2024-09-22T03:17:05.381994+0000","last_fullsized":"2024-09-22T03:17:05.381994+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"63'1834","reported_seq":2074,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-22T03:17:10.049120+0000","last_change":"2024-09-22T03:17:05.165564+0000","last_active":"2024-09-22T03:17:10.049120+0000","last_peered":"2024-09-22T03:17:10.049120+0000","last_clean":"2024-09-22T03:17:10.049120+0000","last_became_active":"2024-09-22T03:17:04.508159+0000","last_became_peered":"2024-09-22T03:17:04.508159+0000","last_unstale":"2024-09-22T03:17:10.049120+0000","last_undegraded":"2024-09-22T03:17:10.049120+0000","last_fullsized":"2024-09-22T03:17:10.049120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66977792,"num_objects":1022,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1834,"num_write_kb":91392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":172,"reported_epoch":63,"state":"active+undersized","last_fresh":"2024-09-22T03:16:37.098707+0000","last_change":"2024-09-22T03:16:37.098332+0000","last_active":"2024-09-22T03:16:37.098707+0000","last_peered":"2024-09-22T03:16:37.098707+0000","last_clean":"2024-09-22T03:15:36.136065+0000","last_became_active":"2024-09-22T03:16:37.098332+0000","last_became_peered":"2024-09-22T03:16:37.098332+0000","last_unstale":"2024-09-22T03:16:37.098707+0000","last_undegraded":"2024-09-22T03:16:37.098707+0000","last_fullsized":"2024-09-22T03:16:37.075367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"63'1784","reported_seq":1829,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-22T03:17:10.048575+0000","last_change":"2024-09-22T03:14:21.212092+0000","last_active":"2024-09-22T03:17:10.048575+0000","last_peered":"2024-09-22T03:17:10.048575+0000","last_clean":"2024-09-22T03:17:10.048575+0000","last_became_active":"2024-09-22T03:14:21.208887+0000","last_became_peered":"2024-09-22T03:14:21.208887+0000","last_unstale":"2024-09-22T03:17:10.048575+0000","last_undegraded":"2024-09-22T03:17:10.048575+0000","last_fullsized":"2024-09-22T03:17:10.048575+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1784,"log_dups_size":0,"ondisk_log_size":1784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62390272,"num_objects":952,"num_object_clones":0,"num_object_copies":2856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1784,"num_write_kb":87552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-22T03:17:10.044150+0000","last_change":"2024-09-22T03:17:05.433256+0000","last_active":"2024-09-22T03:17:10.044150+0000","last_peered":"2024-09-22T03:17:10.044150+0000","last_clean":"2024-09-22T03:17:10.044150+0000","last_became_active":"2024-09-22T03:17:04.435699+0000","last_became_peered":"2024-09-22T03:17:04.435699+0000","last_unstale":"2024-09-22T03:17:10.044150+0000","last_undegraded":"2024-09-22T03:17:10.044150+0000","last_fullsized":"2024-09-22T03:17:10.044150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"63'1702","reported_seq":1730,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-22T03:16:37.218943+0000","last_change":"2024-09-22T03:14:21.175385+0000","last_active":"2024-09-22T03:16:37.218943+0000","last_peered":"2024-09-22T03:16:37.218943+0000","last_clean":"2024-09-22T03:16:37.218943+0000","last_became_active":"2024-09-22T03:14:21.172079+0000","last_became_peered":"2024-09-22T03:14:21.172079+0000","last_unstale":"2024-09-22T03:16:37.218943+0000","last_undegraded":"2024-09-22T03:16:37.218943+0000","last_fullsized":"2024-09-22T03:16:37.218943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1702,"log_dups_size":0,"ondisk_log_size":1702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68878336,"num_objects":1051,"num_object_clones":0,"num_object_copies":3153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1703,"num_write_kb":88128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":104,"reported_epoch":66,"state":"peering","last_fresh":"2024-09-22T03:17:04.377629+0000","last_change":"2024-09-22T03:17:03.388876+0000","last_active":"2024-09-22T03:17:03.388876+0000","last_peered":"2024-09-22T03:16:37.152250+0000","last_clean":"2024-09-22T03:15:36.147591+0000","last_became_active":"2024-09-22T03:16:37.142516+0000","last_became_peered":"2024-09-22T03:16:37.142516+0000","last_unstale":"2024-09-22T03:17:04.377629+0000","last_undegraded":"2024-09-22T03:17:04.377629+0000","last_fullsized":"2024-09-22T03:17:04.377629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":296,"reported_epoch":63,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:16:37.118595+0000","last_change":"2024-09-22T03:16:37.118195+0000","last_active":"2024-09-22T03:16:37.118595+0000","last_peered":"2024-09-22T03:16:37.118595+0000","last_clean":"2024-09-22T03:15:36.145998+0000","last_became_active":"2024-09-22T03:16:37.118195+0000","last_became_peered":"2024-09-22T03:16:37.118195+0000","last_unstale":"2024-09-22T03:16:37.118595+0000","last_undegraded":"2024-09-22T03:16:37.081766+0000","last_fullsized":"2024-09-22T03:16:37.081106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"61'1714","reported_seq":1742,"reported_epoch":69,"state":"peering","last_fresh":"2024-09-22T03:17:08.384895+0000","last_change":"2024-09-22T03:17:03.382865+0000","last_active":"2024-09-22T03:17:03.382865+0000","last_peered":"2024-09-22T03:16:07.745654+0000","last_clean":"2024-09-22T03:16:07.745654+0000","last_became_active":"2024-09-22T03:15:36.205362+0000","last_became_peered":"2024-09-22T03:15:36.205362+0000","last_unstale":"2024-09-22T03:17:08.384895+0000","last_undegraded":"2024-09-22T03:17:08.384895+0000","last_fullsized":"2024-09-22T03:17:08.384895+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1714,"log_dups_size":0,"ondisk_log_size":1714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70647808,"num_objects":1078,"num_object_clones":0,"num_object_copies":3234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1714,"num_write_kb":89344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":69,"state":"peering","last_fresh":"2024-09-22T03:17:08.389012+0000","last_change":"2024-09-22T03:17:03.393622+0000","last_active":"2024-09-22T03:17:03.393622+0000","last_peered":"2024-09-22T03:15:36.225509+0000","last_clean":"2024-09-22T03:15:36.225509+0000","last_became_active":"2024-09-22T03:15:36.222016+0000","last_became_peered":"2024-09-22T03:15:36.222016+0000","last_unstale":"2024-09-22T03:17:08.389012+0000","last_undegraded":"2024-09-22T03:17:08.389012+0000","last_fullsized":"2024-09-22T03:17:08.389012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"61'1683","reported_seq":1718,"reported_epoch":63,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:16:37.170025+0000","last_change":"2024-09-22T03:16:37.166247+0000","last_active":"2024-09-22T03:16:37.170025+0000","last_peered":"2024-09-22T03:16:37.170025+0000","last_clean":"2024-09-22T03:16:07.743111+0000","last_became_active":"2024-09-22T03:16:37.166247+0000","last_became_peered":"2024-09-22T03:16:37.166247+0000","last_unstale":"2024-09-22T03:16:37.170025+0000","last_undegraded":"2024-09-22T03:16:37.086165+0000","last_fullsized":"2024-09-22T03:16:37.085417+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1683,"log_dups_size":0,"ondisk_log_size":1683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73400320,"num_objects":1120,"num_object_clones":0,"num_object_copies":3360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1684,"num_write_kb":89728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":1120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":183,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-22T03:17:10.049895+0000","last_change":"2024-09-22T03:14:19.150234+0000","last_active":"2024-09-22T03:17:10.049895+0000","last_peered":"2024-09-22T03:17:10.049895+0000","last_clean":"2024-09-22T03:17:10.049895+0000","last_became_active":"2024-09-22T03:14:07.140728+0000","last_became_peered":"2024-09-22T03:14:07.140728+0000","last_unstale":"2024-09-22T03:17:10.049895+0000","last_undegraded":"2024-09-22T03:17:10.049895+0000","last_fullsized":"2024-09-22T03:17:10.049895+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"63'1747","reported_seq":1784,"reported_epoch":63,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:16:37.123335+0000","last_change":"2024-09-22T03:16:37.110336+0000","last_active":"2024-09-22T03:16:37.123335+0000","last_peered":"2024-09-22T03:16:37.123335+0000","last_clean":"2024-09-22T03:16:07.823189+0000","last_became_active":"2024-09-22T03:16:37.110336+0000","last_became_peered":"2024-09-22T03:16:37.110336+0000","last_unstale":"2024-09-22T03:16:37.123335+0000","last_undegraded":"2024-09-22T03:16:37.081203+0000","last_fullsized":"2024-09-22T03:16:37.080520+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1747,"log_dups_size":0,"ondisk_log_size":1747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71434240,"num_objects":1090,"num_object_clones":0,"num_object_copies":3270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1748,"num_write_kb":90816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1090}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":145,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-22T03:17:10.049086+0000","last_change":"2024-09-22T03:17:05.388172+0000","last_active":"2024-09-22T03:17:10.049086+0000","last_peered":"2024-09-22T03:17:10.049086+0000","last_clean":"2024-09-22T03:17:10.049086+0000","last_became_active":"2024-09-22T03:17:04.435070+0000","last_became_peered":"2024-09-22T03:17:04.435070+0000","last_unstale":"2024-09-22T03:17:10.049086+0000","last_undegraded":"2024-09-22T03:17:10.049086+0000","last_fullsized":"2024-09-22T03:17:10.049086+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'1881","reported_seq":1919,"reported_epoch":69,"state":"activating","last_fresh":"2024-09-22T03:17:10.046059+0000","last_change":"2024-09-22T03:17:08.390996+0000","last_active":"2024-09-22T03:17:08.390110+0000","last_peered":"2024-09-22T03:17:06.504951+0000","last_clean":"2024-09-22T03:16:07.793409+0000","last_became_active":"2024-09-22T03:17:05.499979+0000","last_became_peered":"2024-09-22T03:17:05.499979+0000","last_unstale":"2024-09-22T03:17:10.046059+0000","last_undegraded":"2024-09-22T03:17:10.046059+0000","last_fullsized":"2024-09-22T03:17:10.046059+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":1881,"log_dups_size":0,"ondisk_log_size":1881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65863680,"num_objects":1005,"num_object_clones":0,"num_object_copies":3015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1881,"num_write_kb":92352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":1005}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":128,"reported_epoch":69,"state":"peering","last_fresh":"2024-09-22T03:17:08.385829+0000","last_change":"2024-09-22T03:17:03.382154+0000","last_active":"2024-09-22T03:17:03.382154+0000","last_peered":"2024-09-22T03:15:36.182916+0000","last_clean":"2024-09-22T03:15:36.182916+0000","last_became_active":"2024-09-22T03:15:36.173982+0000","last_became_peered":"2024-09-22T03:15:36.173982+0000","last_unstale":"2024-09-22T03:17:08.385829+0000","last_undegraded":"2024-09-22T03:17:08.385829+0000","last_fullsized":"2024-09-22T03:17:08.385829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":1089339416,"num_objects":16623,"num_object_clones":0,"num_object_copies":49869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":28075,"num_write_kb":1430273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201744384,"data_stored":3227713608,"data_compressed":153269106,"data_compressed_allocated":201732096,"data_compressed_original":3227713536,"omap_allocated":0,"internal_metadata":0},"log_size":28071,"ondisk_log_size":28071,"up":45,"acting":45,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":23,"acting":23,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":53,"seq":227633266867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":112732,"kb_used_data":51788,"kb_used_omap":0,"kb_used_meta":60928,"kb_avail":94259108,"statfs":{"total":96636764160,"available":96521326592,"internally_reserved":0,"allocated":53030912,"data_stored":839988909,"data_compressed":39877166,"data_compressed_allocated":52486144,"data_compressed_original":839778304,"omap_allocated":0,"internal_metadata":62390272},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673329,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":111344,"kb_used_data":40096,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94260496,"statfs":{"total":96636764160,"available":96522747904,"internally_reserved":0,"allocated":41058304,"data_stored":647457951,"data_compressed":30736074,"data_compressed_allocated":40456192,"data_compressed_original":647217152,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":64,"seq":274877906954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93856,"kb_used_data":55696,"kb_used_omap":0,"kb_used_meta":38144,"kb_avail":94277984,"statfs":{"total":96636764160,"available":96540655616,"internally_reserved":0,"allocated":57032704,"data_stored":903048351,"data_compressed":42872875,"data_compressed_allocated":56430592,"data_compressed_original":902807552,"omap_allocated":0,"internal_metadata":39059456},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123124,"kb_used_data":51876,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94248716,"statfs":{"total":96636764160,"available":96510685184,"internally_reserved":0,"allocated":53121024,"data_stored":841420575,"data_compressed":39947595,"data_compressed_allocated":52580352,"data_compressed_original":841203712,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52445184,"data_stored":839057432,"data_compressed":39842935,"data_compressed_allocated":52441088,"data_compressed_original":839057408,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56393728,"data_stored":902234136,"data_compressed":42842903,"data_compressed_allocated":56389632,"data_compressed_original":902234112,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40419328,"data_stored":646643736,"data_compressed":30706102,"data_compressed_allocated":40415232,"data_compressed_original":646643712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":52486144,"data_stored":839778304,"data_compressed":39877166,"data_compressed_allocated":52486144,"data_compressed_original":839778304,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:17:24.218 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 3] live_osds: [2, 1] 2024-09-22T03:17:24.218 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:17:24.218 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:17:24.219 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:17:24.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:24.331+0000 7f376f0ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:24.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:24.339+0000 7f376f0ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:33.060 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:33.202 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:33.206 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:33.206 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:34.345 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:34.348 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:34.348 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:17:34.349 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:17:34.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:34.453+0000 7f1244797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:34.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:34.462+0000 7f1244797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:34.510 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:17:34.510 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:17:34.510 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:17:34.510 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:17:34.510 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:17:34.510 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:17:34.511 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:17:34.511 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:17:34.511 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:17:34.511 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:17:34.511 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:17:34.511 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:17:34.514 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:17:34.514 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:17:34.514 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:17:34.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:34.620+0000 7ff11962c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:34.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:34.629+0000 7ff11962c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:34.645 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:17:34.645 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:17:34.645 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:17:34.645 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:17:34.645 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:17:34.645 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:17:34.645 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:17:34.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:17:34.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:17:34.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:17:34.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:17:34.646 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:17:34.648 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:17:34.649 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:17:34.649 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:17:34.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:34.753+0000 7fd309f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:34.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:34.762+0000 7fd309f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:34.768 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:34.772 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:34.773 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:17:35.206 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:17:35.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:35.306+0000 7faf7a397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:35.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:35.310+0000 7faf7a397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:35.315 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:35.345 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:35.348 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:35.348 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:37.349 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:17:37.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:37.462+0000 7fa42bd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:37.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:37.467+0000 7fa42bd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:37.472 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:37.502 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:37.505 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:37.506 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:39.506 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:17:39.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:39.617+0000 7fcffaa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:39.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:39.622+0000 7fcffaa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:39.629 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:39.658 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:39.662 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:39.662 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:39.773 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 3] live_osds: [2, 1] 2024-09-22T03:17:39.773 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:17:39.773 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-22T03:17:39.774 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:17:39.774 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:17:39.776 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:17:39.777 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:17:39.875 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:17:39.878 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:17:39.878 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:17:41.663 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:17:41.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:41.766+0000 7ff40986c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:41.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:41.771+0000 7ff40986c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:41.778 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:41.816 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:41.818 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:41.819 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:43.820 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:17:43.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:43.922+0000 7fa3e6a63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:43.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:43.927+0000 7fa3e6a63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:43.934 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:43.968 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:43.974 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:43.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:44.879 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:17:44.976 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:17:44.978 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:17:44.979 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:17:45.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:17:46.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:46.107+0000 7fa8ada23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:46.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:46.112+0000 7fa8ada23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:46.119 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:46.158 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:46.159 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:48.160 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:17:48.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:48.260+0000 7fbdfaa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:48.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:48.264+0000 7fbdfaa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:48.271 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:48.300 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:48.303 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:48.303 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:49.980 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:17:50.085 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:17:50.088 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:17:50.089 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:17:50.304 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:17:50.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:50.405+0000 7fb7028a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:50.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:50.410+0000 7fb7028a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:50.416 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:50.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:50.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:52.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:17:52.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:52.557+0000 7f58b9cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:52.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:52.562+0000 7f58b9cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:52.568 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:52.635 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:52.639 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:52.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:54.640 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:17:54.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:54.740+0000 7fb6e8157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:54.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:54.745+0000 7fb6e8157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:54.751 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:17:54.779 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:54.783 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:54.783 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:55.089 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 47, 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2", 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "4.7", 2024-09-22T03:17:55.225 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 75, 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 75, 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 75 epoch_requested: 75 MQuery 4.7 from 2 query_epoch 75 query: query(info 0'0 epoch_sent 75)", 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.202689+0000" 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.202691+0000" 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.210097+0000" 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.227 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.212200+0000" 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:17:55.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.212346+0000" 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.214308+0000" 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.215578+0000" 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:17:55.231 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 51, 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2", 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "1.5", 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 75, 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 75, 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 75 epoch_requested: 75 MQuery 1.5 from 2 query_epoch 75 query: query(info 0'0 epoch_sent 75)", 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.211840+0000" 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.211843+0000" 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.214360+0000" 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.215523+0000" 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.234 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.215669+0000" 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.216522+0000" 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "peering_event", 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 52, 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "PeeringEvent": { 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "1", 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "4.3", 2024-09-22T03:17:55.238 INFO:teuthology.orchestra.run.smithi146.stdout: "sent": 75, 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "requested": 75, 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "evt": "epoch_sent: 75 epoch_requested: 75 MNotifyRec 4.3 from 1 notify: (query:75 sent:75 4.3( v 74'1843 (0'0,74'1843] local-lis/les=73/74 n=995 ec=56/56 lis/c=73/60 les/c/f=74/61/0 sis=75 pruub=11.003258705s) ([60,74] all_participants={0, 1, 2} intervals=[([62,63] acting={0, 2}), ([73,74] acting={1, 2})])) features: 0x3f03cffffffdffff +create_info", 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "events": [ 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.214136+0000" 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.214140+0000" 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.214580+0000" 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.216378+0000" 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.217637+0000" 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 15, 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.244 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096097+0000" 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096122+0000" 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.107121+0000" 2024-09-22T03:17:55.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.107406+0000" 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.205299+0000" 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.206322+0000" 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.247 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.206757+0000" 2024-09-22T03:17:55.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:17:55.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.207071+0000" 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.207209+0000", 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PGActivationBlocker", 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "4.0" 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:17:55.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.253 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16, 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:17:55.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096361+0000" 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096374+0000" 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.107577+0000" 2024-09-22T03:17:55.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.206215+0000" 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.208702+0000" 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.259 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.209096+0000" 2024-09-22T03:17:55.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:17:55.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.209254+0000" 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.209336+0000", 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "PGActivationBlocker", 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "4.6" 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.264 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 18, 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:17:55.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096606+0000" 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096609+0000" 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.208393+0000" 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.208563+0000" 2024-09-22T03:17:55.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.210140+0000" 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:17:55.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.210911+0000", 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "ClientRequest::PGPipeline::await_map", 2024-09-22T03:17:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 17 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:17:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 17, 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096484+0000" 2024-09-22T03:17:55.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.096487+0000" 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.206663+0000" 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.208077+0000" 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.282 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.210044+0000" 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.210491+0000" 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.284 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:17:55.211004+0000", 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "blocker": { 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-22T03:17:55.285 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "waiting": 1, 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "held_by_operation_id": 15 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.286 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.289 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:17:55.292 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:17:56.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:17:56.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:56.915+0000 7efebfcb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:56.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:56.921+0000 7efebea50640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:57.187 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:57.190 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:57.191 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:17:59.192 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:17:59.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:59.319+0000 7fd74bb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:59.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:17:59.328+0000 7fd74a8b5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:17:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:17:59.619 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:17:59.619 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:00.294 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [2, 1, 0] 2024-09-22T03:18:00.294 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:18:00.294 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:18:00.295 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:18:00.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:00.429+0000 7f940bb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:00.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:00.447+0000 7f940a8f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:00.543 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:18:00.543 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:18:00.543 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:18:00.544 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:18:00.548 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:00.550 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:00.550 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:18:00.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:00.690+0000 7f56a8597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:00.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:00.701+0000 7f56a8597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:00.760 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:18:00.760 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:18:00.760 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:18:00.761 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:18:00.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:00.764 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:00.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:18:00.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:00.896+0000 7fb90d197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:00.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:00.905+0000 7fb90d197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:18:00.930 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:18:00.931 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:18:00.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:00.934 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:00.934 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:18:01.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:01.079+0000 7fd23566c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:01.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:01.092+0000 7fd23566c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:01.101 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:01.104 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:01.105 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:01.620 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:01.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:01.769+0000 7fa742cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:01.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:01.777+0000 7fa742cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:02.143 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:02.147 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:02.148 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:04.149 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:04.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:04.275+0000 7fc7ab30e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:04.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:04.281+0000 7fc7ab30e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:04.562 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:04.566 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:04.566 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:06.105 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [2, 1, 0] 2024-09-22T03:18:06.184 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:18:06.184 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-22T03:18:06.184 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:18:06.185 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:18:06.187 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:18:06.188 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:18:06.320 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:18:06.324 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:06.325 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:18:06.566 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:06.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:06.715+0000 7f51dd66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:06.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:06.723+0000 7f51dd66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:06.986 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:06.989 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:06.990 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:08.990 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:09.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:09.114+0000 7f048946c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:09.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:09.121+0000 7f048946c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:09.430 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:09.435 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:09.435 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:11.326 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:18:11.436 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:11.452 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:18:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:11.456 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:18:11.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:11.559+0000 7f80a5b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:11.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:11.568+0000 7f80a5b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:11.856 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:11.860 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:11.860 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:13.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:14.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:14.007+0000 7ff15d8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:14.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:14.015+0000 7ff15d8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:14.249 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:14.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:14.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:16.255 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:16.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:16.391+0000 7fddf7157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:16.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:16.397+0000 7fddf7157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:16.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:18:16.558 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:18:16.561 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:16.561 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:18:16.618 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:16.621 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:16.622 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:18.623 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:18.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:18.763+0000 7f42d2ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:18.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:18.771+0000 7f42d2ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:19.002 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:19.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:19.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:21.007 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:21.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:21.130+0000 7f7b1e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:21.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:21.137+0000 7f7b1e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:21.386 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:21.390 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:21.390 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:21.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:18:21.683 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:18:21.687 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:21.687 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:18:23.392 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:23.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:23.514+0000 7ff20aad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:23.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:23.521+0000 7ff20aad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:23.806 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:18:23.809 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:18:23.809 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:18:25.811 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:25.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:25.915+0000 7fbc9ca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:25.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:25.919+0000 7fbc9ca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:26.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:18:26.789 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:18:26.789 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [] 2024-09-22T03:18:26.790 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:18:27.996 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:28.156 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:28.153+0000 7f51f3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:28.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:28.159+0000 7f51f3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:30.340 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:30.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:30.462+0000 7f28ff8ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:30.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:30.467+0000 7f28ff8ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:31.798 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:18:31.799 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:18:31.799 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:18:31.799 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:18:31.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:31.917+0000 7fdd286b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:31.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:31.927+0000 7fdd286b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:32.027 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:18:32.027 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:18:32.027 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:18:32.027 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:18:32.027 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:18:32.027 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:18:32.027 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:18:32.028 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:18:32.028 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:18:32.028 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:18:32.028 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:18:32.028 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:18:32.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:32.031 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:32.032 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:18:32.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:32.163+0000 7f8093ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:32.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:32.174+0000 7f8093ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:32.245 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:18:32.245 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:18:32.245 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:18:32.245 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:18:32.245 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:18:32.245 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:18:32.246 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:18:32.246 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:18:32.246 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:18:32.246 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:18:32.246 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:18:32.246 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:18:32.249 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:32.249 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:32.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:18:32.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:32.377+0000 7fab77797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:32.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:32.389+0000 7fab77797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:32.455 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:18:32.455 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:18:32.455 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:18:32.455 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:18:32.455 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:18:32.455 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:18:32.455 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:18:32.456 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:18:32.456 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:18:32.456 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:18:32.456 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:18:32.456 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:18:32.459 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:32.459 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:32.460 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:18:32.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:32.578+0000 7f5591f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:32.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:32.589+0000 7f5591f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:18:32.611 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:18:32.612 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:18:32.612 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:18:32.612 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:18:32.612 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:18:32.614 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:18:32.615 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:18:32.950 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:33.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:33.073+0000 7f8eba197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:33.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:33.080+0000 7f8eba197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:35.638 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:35.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:35.755+0000 7f32c0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:35.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:35.761+0000 7f32c0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:37.616 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:18:37.616 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:18:37.616 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:18:37.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:37.742+0000 7fd545597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:37.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:37.755+0000 7fd545597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:38.030 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:38.051 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:18:38.051 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:18:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-22T03:18:37.320112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":412615243,"num_objects":6292,"num_object_clones":0,"num_object_copies":18876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":38549,"num_write_kb":1432169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1897,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38497,"ondisk_log_size":38497,"up":75,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":321768,"kb_used_data":101092,"kb_used_omap":0,"kb_used_meta":220608,"kb_avail":377165592,"statfs":{"total":386547056640,"available":386217566208,"internally_reserved":0,"allocated":103518208,"data_stored":1611260185,"data_compressed":76464616,"data_compressed_allocated":100646912,"data_compressed_original":1610104832,"omap_allocated":0,"internal_metadata":225902592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129761280,"num_objects":-1980,"num_object_clones":0,"num_object_copies":-5940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1980,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002020"},"pg_stats":[{"pgid":"4.f","version":"84'2324","reported_seq":2433,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.184033+0000","last_change":"2024-09-22T03:18:27.812526+0000","last_active":"2024-09-22T03:18:33.184033+0000","last_peered":"2024-09-22T03:18:33.184033+0000","last_clean":"2024-09-22T03:17:25.054857+0000","last_became_active":"2024-09-22T03:18:27.396103+0000","last_became_peered":"2024-09-22T03:18:27.396103+0000","last_unstale":"2024-09-22T03:18:33.184033+0000","last_undegraded":"2024-09-22T03:18:27.805960+0000","last_fullsized":"2024-09-22T03:18:27.137870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2324,"log_dups_size":0,"ondisk_log_size":2324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":373,"num_object_clones":0,"num_object_copies":1119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2325,"num_write_kb":86336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":373}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"84'2458","reported_seq":2759,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.274996+0000","last_change":"2024-09-22T03:18:27.484558+0000","last_active":"2024-09-22T03:18:33.274996+0000","last_peered":"2024-09-22T03:18:33.274996+0000","last_clean":"2024-09-22T03:16:07.669976+0000","last_became_active":"2024-09-22T03:18:27.478074+0000","last_became_peered":"2024-09-22T03:18:27.478074+0000","last_unstale":"2024-09-22T03:18:33.274996+0000","last_undegraded":"2024-09-22T03:18:27.135012+0000","last_fullsized":"2024-09-22T03:18:27.100436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27066368,"num_objects":413,"num_object_clones":0,"num_object_copies":1239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2459,"num_write_kb":91904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":413}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"84'2394","reported_seq":2578,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:33.483488+0000","last_change":"2024-09-22T03:17:58.034056+0000","last_active":"2024-09-22T03:18:33.483488+0000","last_peered":"2024-09-22T03:18:33.483488+0000","last_clean":"2024-09-22T03:18:33.483488+0000","last_became_active":"2024-09-22T03:17:56.250038+0000","last_became_peered":"2024-09-22T03:17:56.250038+0000","last_unstale":"2024-09-22T03:18:33.483488+0000","last_undegraded":"2024-09-22T03:18:33.483488+0000","last_fullsized":"2024-09-22T03:18:33.483488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2394,"log_dups_size":0,"ondisk_log_size":2394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641560,"num_objects":377,"num_object_clones":0,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2395,"num_write_kb":88641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"84'2347","reported_seq":2879,"reported_epoch":84,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:18:35.652260+0000","last_change":"2024-09-22T03:18:29.455732+0000","last_active":"2024-09-22T03:18:35.652260+0000","last_peered":"2024-09-22T03:18:35.652260+0000","last_clean":"2024-09-22T03:16:07.774677+0000","last_became_active":"2024-09-22T03:18:27.514659+0000","last_became_peered":"2024-09-22T03:18:27.514659+0000","last_unstale":"2024-09-22T03:18:35.652260+0000","last_undegraded":"2024-09-22T03:18:35.652260+0000","last_fullsized":"2024-09-22T03:18:27.105388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23396352,"num_objects":357,"num_object_clones":0,"num_object_copies":1071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2347,"num_write_kb":86528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":357}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"84'2442","reported_seq":2519,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:35.506904+0000","last_change":"2024-09-22T03:17:56.454442+0000","last_active":"2024-09-22T03:18:35.506904+0000","last_peered":"2024-09-22T03:18:35.506904+0000","last_clean":"2024-09-22T03:18:35.506904+0000","last_became_active":"2024-09-22T03:17:56.225997+0000","last_became_peered":"2024-09-22T03:17:56.225997+0000","last_unstale":"2024-09-22T03:18:35.506904+0000","last_undegraded":"2024-09-22T03:18:35.506904+0000","last_fullsized":"2024-09-22T03:18:35.506904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2442,"log_dups_size":0,"ondisk_log_size":2442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23003136,"num_objects":351,"num_object_clones":0,"num_object_copies":1053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":89408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"84'2453","reported_seq":2529,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.554660+0000","last_change":"2024-09-22T03:18:27.782630+0000","last_active":"2024-09-22T03:18:33.554660+0000","last_peered":"2024-09-22T03:18:33.554660+0000","last_clean":"2024-09-22T03:16:07.663835+0000","last_became_active":"2024-09-22T03:18:27.776960+0000","last_became_peered":"2024-09-22T03:18:27.776960+0000","last_unstale":"2024-09-22T03:18:33.554660+0000","last_undegraded":"2024-09-22T03:18:27.338410+0000","last_fullsized":"2024-09-22T03:18:27.296011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28573696,"num_objects":436,"num_object_clones":0,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2454,"num_write_kb":92480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":436}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"84'2333","reported_seq":3094,"reported_epoch":84,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:18:33.558696+0000","last_change":"2024-09-22T03:18:27.815412+0000","last_active":"2024-09-22T03:18:33.558696+0000","last_peered":"2024-09-22T03:18:33.558696+0000","last_clean":"2024-09-22T03:16:07.756885+0000","last_became_active":"2024-09-22T03:18:27.474936+0000","last_became_peered":"2024-09-22T03:18:27.474936+0000","last_unstale":"2024-09-22T03:18:33.558696+0000","last_undegraded":"2024-09-22T03:18:33.558696+0000","last_fullsized":"2024-09-22T03:18:27.102547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2333,"log_dups_size":0,"ondisk_log_size":2333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":411,"num_object_clones":0,"num_object_copies":1233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2333,"num_write_kb":87808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":411}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"84'2310","reported_seq":2467,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.478997+0000","last_change":"2024-09-22T03:18:27.757469+0000","last_active":"2024-09-22T03:18:33.478997+0000","last_peered":"2024-09-22T03:18:33.478997+0000","last_clean":"2024-09-22T03:16:07.676059+0000","last_became_active":"2024-09-22T03:18:27.753769+0000","last_became_peered":"2024-09-22T03:18:27.753769+0000","last_unstale":"2024-09-22T03:18:33.478997+0000","last_undegraded":"2024-09-22T03:18:27.441737+0000","last_fullsized":"2024-09-22T03:18:27.339846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":411,"num_object_clones":0,"num_object_copies":1233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2311,"num_write_kb":87104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":411}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"84'2428","reported_seq":2593,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.506508+0000","last_change":"2024-09-22T03:18:27.767849+0000","last_active":"2024-09-22T03:18:33.506508+0000","last_peered":"2024-09-22T03:18:33.506508+0000","last_clean":"2024-09-22T03:16:07.777614+0000","last_became_active":"2024-09-22T03:18:27.756267+0000","last_became_peered":"2024-09-22T03:18:27.756267+0000","last_unstale":"2024-09-22T03:18:33.506508+0000","last_undegraded":"2024-09-22T03:18:27.339928+0000","last_fullsized":"2024-09-22T03:18:27.293928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2428,"log_dups_size":0,"ondisk_log_size":2428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26673152,"num_objects":407,"num_object_clones":0,"num_object_copies":1221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2429,"num_write_kb":90752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":407}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.073258+0000","last_change":"2024-09-22T03:18:27.105150+0000","last_active":"2024-09-22T03:18:30.073258+0000","last_peered":"2024-09-22T03:18:30.073258+0000","last_clean":"2024-09-22T03:18:30.073258+0000","last_became_active":"2024-09-22T03:18:26.172226+0000","last_became_peered":"2024-09-22T03:18:26.172226+0000","last_unstale":"2024-09-22T03:18:30.073258+0000","last_undegraded":"2024-09-22T03:18:30.073258+0000","last_fullsized":"2024-09-22T03:18:30.073258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"84'2427","reported_seq":2708,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.212792+0000","last_change":"2024-09-22T03:18:27.528201+0000","last_active":"2024-09-22T03:18:33.212792+0000","last_peered":"2024-09-22T03:18:33.212792+0000","last_clean":"2024-09-22T03:17:25.053887+0000","last_became_active":"2024-09-22T03:18:27.517015+0000","last_became_peered":"2024-09-22T03:18:27.517015+0000","last_unstale":"2024-09-22T03:18:33.212792+0000","last_undegraded":"2024-09-22T03:18:27.200533+0000","last_fullsized":"2024-09-22T03:18:27.170931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2427,"log_dups_size":0,"ondisk_log_size":2427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28049408,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":514,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2428,"num_write_kb":91392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":207,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.078077+0000","last_change":"2024-09-22T03:17:57.159552+0000","last_active":"2024-09-22T03:18:30.078077+0000","last_peered":"2024-09-22T03:18:30.078077+0000","last_clean":"2024-09-22T03:18:30.078077+0000","last_became_active":"2024-09-22T03:17:56.232475+0000","last_became_peered":"2024-09-22T03:17:56.232475+0000","last_unstale":"2024-09-22T03:18:30.078077+0000","last_undegraded":"2024-09-22T03:18:30.078077+0000","last_fullsized":"2024-09-22T03:18:30.078077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"84'2341","reported_seq":2442,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.248237+0000","last_change":"2024-09-22T03:18:27.839938+0000","last_active":"2024-09-22T03:18:33.248237+0000","last_peered":"2024-09-22T03:18:33.248237+0000","last_clean":"2024-09-22T03:17:25.053970+0000","last_became_active":"2024-09-22T03:18:27.515847+0000","last_became_peered":"2024-09-22T03:18:27.515847+0000","last_unstale":"2024-09-22T03:18:33.248237+0000","last_undegraded":"2024-09-22T03:18:27.834340+0000","last_fullsized":"2024-09-22T03:18:27.137958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2341,"log_dups_size":0,"ondisk_log_size":2341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":394,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":87552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":394}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":242,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.097176+0000","last_change":"2024-09-22T03:18:27.142593+0000","last_active":"2024-09-22T03:18:30.097176+0000","last_peered":"2024-09-22T03:18:30.097176+0000","last_clean":"2024-09-22T03:18:30.097176+0000","last_became_active":"2024-09-22T03:18:26.201180+0000","last_became_peered":"2024-09-22T03:18:26.201180+0000","last_unstale":"2024-09-22T03:18:30.097176+0000","last_undegraded":"2024-09-22T03:18:30.097176+0000","last_fullsized":"2024-09-22T03:18:30.097176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"84'2377","reported_seq":2441,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:35.643953+0000","last_change":"2024-09-22T03:18:27.760703+0000","last_active":"2024-09-22T03:18:35.643953+0000","last_peered":"2024-09-22T03:18:35.643953+0000","last_clean":"2024-09-22T03:16:41.319459+0000","last_became_active":"2024-09-22T03:18:27.408241+0000","last_became_peered":"2024-09-22T03:18:27.408241+0000","last_unstale":"2024-09-22T03:18:35.643953+0000","last_undegraded":"2024-09-22T03:18:27.757893+0000","last_fullsized":"2024-09-22T03:18:27.164516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641536,"num_objects":376,"num_object_clones":0,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2378,"num_write_kb":88128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":376}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.074430+0000","last_change":"2024-09-22T03:18:27.105253+0000","last_active":"2024-09-22T03:18:30.074430+0000","last_peered":"2024-09-22T03:18:30.074430+0000","last_clean":"2024-09-22T03:18:30.074430+0000","last_became_active":"2024-09-22T03:18:26.176249+0000","last_became_peered":"2024-09-22T03:18:26.176249+0000","last_unstale":"2024-09-22T03:18:30.074430+0000","last_undegraded":"2024-09-22T03:18:30.074430+0000","last_fullsized":"2024-09-22T03:18:30.074430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":330,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.079184+0000","last_change":"2024-09-22T03:17:56.230173+0000","last_active":"2024-09-22T03:18:30.079184+0000","last_peered":"2024-09-22T03:18:30.079184+0000","last_clean":"2024-09-22T03:18:30.079184+0000","last_became_active":"2024-09-22T03:17:56.222228+0000","last_became_peered":"2024-09-22T03:17:56.222228+0000","last_unstale":"2024-09-22T03:18:30.079184+0000","last_undegraded":"2024-09-22T03:18:30.079184+0000","last_fullsized":"2024-09-22T03:18:30.079184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"84'2395","reported_seq":2885,"reported_epoch":84,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:18:33.578336+0000","last_change":"2024-09-22T03:18:27.894267+0000","last_active":"2024-09-22T03:18:33.578336+0000","last_peered":"2024-09-22T03:18:33.578336+0000","last_clean":"2024-09-22T03:16:07.745654+0000","last_became_active":"2024-09-22T03:18:27.761699+0000","last_became_peered":"2024-09-22T03:18:27.761699+0000","last_unstale":"2024-09-22T03:18:33.578336+0000","last_undegraded":"2024-09-22T03:18:33.578336+0000","last_fullsized":"2024-09-22T03:18:27.198426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2395,"log_dups_size":0,"ondisk_log_size":2395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26017792,"num_objects":397,"num_object_clones":0,"num_object_copies":1191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2395,"num_write_kb":89344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":397}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.084737+0000","last_change":"2024-09-22T03:17:57.157733+0000","last_active":"2024-09-22T03:18:30.084737+0000","last_peered":"2024-09-22T03:18:30.084737+0000","last_clean":"2024-09-22T03:18:30.084737+0000","last_became_active":"2024-09-22T03:17:56.209198+0000","last_became_peered":"2024-09-22T03:17:56.209198+0000","last_unstale":"2024-09-22T03:18:30.084737+0000","last_undegraded":"2024-09-22T03:18:30.084737+0000","last_fullsized":"2024-09-22T03:18:30.084737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"84'2424","reported_seq":2693,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:35.599031+0000","last_change":"2024-09-22T03:18:27.728711+0000","last_active":"2024-09-22T03:18:35.599031+0000","last_peered":"2024-09-22T03:18:35.599031+0000","last_clean":"2024-09-22T03:16:07.743111+0000","last_became_active":"2024-09-22T03:18:27.724438+0000","last_became_peered":"2024-09-22T03:18:27.724438+0000","last_unstale":"2024-09-22T03:18:35.599031+0000","last_undegraded":"2024-09-22T03:18:27.163677+0000","last_fullsized":"2024-09-22T03:18:27.136744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2424,"log_dups_size":0,"ondisk_log_size":2424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":379,"num_object_clones":0,"num_object_copies":1137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2425,"num_write_kb":89728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":229,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.105888+0000","last_change":"2024-09-22T03:18:27.203806+0000","last_active":"2024-09-22T03:18:30.105888+0000","last_peered":"2024-09-22T03:18:30.105888+0000","last_clean":"2024-09-22T03:18:30.105888+0000","last_became_active":"2024-09-22T03:18:26.256641+0000","last_became_peered":"2024-09-22T03:18:26.256641+0000","last_unstale":"2024-09-22T03:18:30.105888+0000","last_undegraded":"2024-09-22T03:18:30.105888+0000","last_fullsized":"2024-09-22T03:18:30.105888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"84'2473","reported_seq":2550,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:35.556080+0000","last_change":"2024-09-22T03:17:56.430641+0000","last_active":"2024-09-22T03:18:35.556080+0000","last_peered":"2024-09-22T03:18:35.556080+0000","last_clean":"2024-09-22T03:18:35.556080+0000","last_became_active":"2024-09-22T03:17:56.210584+0000","last_became_peered":"2024-09-22T03:17:56.210584+0000","last_unstale":"2024-09-22T03:18:35.556080+0000","last_undegraded":"2024-09-22T03:18:35.556080+0000","last_fullsized":"2024-09-22T03:18:35.556080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2473,"log_dups_size":0,"ondisk_log_size":2473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855104,"num_objects":364,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2474,"num_write_kb":90816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":189,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.090890+0000","last_change":"2024-09-22T03:18:27.456830+0000","last_active":"2024-09-22T03:18:30.090890+0000","last_peered":"2024-09-22T03:18:30.090890+0000","last_clean":"2024-09-22T03:18:30.090890+0000","last_became_active":"2024-09-22T03:18:26.167950+0000","last_became_peered":"2024-09-22T03:18:26.167950+0000","last_unstale":"2024-09-22T03:18:30.090890+0000","last_undegraded":"2024-09-22T03:18:30.090890+0000","last_fullsized":"2024-09-22T03:18:30.090890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"84'2471","reported_seq":2555,"reported_epoch":84,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:18:33.547708+0000","last_change":"2024-09-22T03:18:27.782364+0000","last_active":"2024-09-22T03:18:33.547708+0000","last_peered":"2024-09-22T03:18:33.547708+0000","last_clean":"2024-09-22T03:16:07.793409+0000","last_became_active":"2024-09-22T03:18:27.777982+0000","last_became_peered":"2024-09-22T03:18:27.777982+0000","last_unstale":"2024-09-22T03:18:33.547708+0000","last_undegraded":"2024-09-22T03:18:27.453689+0000","last_fullsized":"2024-09-22T03:18:27.341273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:14:20.110058+0000","last_clean_scrub_stamp":"2024-09-22T03:14:20.110058+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":414,"num_object_clones":0,"num_object_copies":1242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2472,"num_write_kb":92352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":414}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:18:30.077037+0000","last_change":"2024-09-22T03:18:27.300161+0000","last_active":"2024-09-22T03:18:30.077037+0000","last_peered":"2024-09-22T03:18:30.077037+0000","last_clean":"2024-09-22T03:18:30.077037+0000","last_became_active":"2024-09-22T03:18:26.168994+0000","last_became_peered":"2024-09-22T03:18:26.168994+0000","last_unstale":"2024-09-22T03:18:30.077037+0000","last_undegraded":"2024-09-22T03:18:30.077037+0000","last_fullsized":"2024-09-22T03:18:30.077037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":412024856,"num_objects":6288,"num_object_clones":0,"num_object_copies":18864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":38410,"num_write_kb":1430273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1897,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108601344,"data_stored":1737424968,"data_compressed":82502228,"data_compressed_allocated":108589056,"data_compressed_original":1737424896,"omap_allocated":0,"internal_metadata":0},"log_size":38397,"ondisk_log_size":38397,"up":48,"acting":35,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":81,"seq":347892350986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86532,"kb_used_data":43124,"kb_used_omap":0,"kb_used_meta":43392,"kb_avail":94285308,"statfs":{"total":96636764160,"available":96548155392,"internally_reserved":0,"allocated":44158976,"data_stored":695613089,"data_compressed":33018318,"data_compressed_allocated":43458560,"data_compressed_original":695336960,"omap_allocated":0,"internal_metadata":44433408},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673409,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110860,"kb_used_data":17400,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94260980,"statfs":{"total":96636764160,"available":96523243520,"internally_reserved":0,"allocated":17817600,"data_stored":273692889,"data_compressed":12985227,"data_compressed_allocated":17092608,"data_compressed_original":273399808,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":64,"seq":274877907034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64312,"kb_used_data":21992,"kb_used_omap":0,"kb_used_meta":42304,"kb_avail":94307528,"statfs":{"total":96636764160,"available":96570908672,"internally_reserved":0,"allocated":22519808,"data_stored":348928217,"data_compressed":16557803,"data_compressed_allocated":21794816,"data_compressed_original":348635136,"omap_allocated":0,"internal_metadata":43319296},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":75,"seq":322122547240,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60064,"kb_used_data":18576,"kb_used_omap":0,"kb_used_meta":41472,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":19021824,"data_stored":293025990,"data_compressed":13903268,"data_compressed_allocated":18300928,"data_compressed_original":292732928,"omap_allocated":0,"internal_metadata":42467328},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17883136,"data_stored":286064664,"data_compressed":13583880,"data_compressed_allocated":17879040,"data_compressed_original":286064640,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21618688,"data_stored":345833496,"data_compressed":16422023,"data_compressed_allocated":21614592,"data_compressed_original":345833472,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16613376,"data_stored":265748504,"data_compressed":12619159,"data_compressed_allocated":16609280,"data_compressed_original":265748480,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":52486144,"data_stored":839778304,"data_compressed":39877166,"data_compressed_allocated":52486144,"data_compressed_original":839778304,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:18:38.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:38.190+0000 7ff9c7c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:38.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:38.196+0000 7ff9c7c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:40.476 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:18:40.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:40.585+0000 7f997ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:40.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:40.591+0000 7f997ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:42.909 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:18:43.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:43.035+0000 7f6acb957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:43.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:18:43.042+0000 7f6acb957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:18:43.077 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:18:43.077 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:18:43.077 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [2, 1, 0, 3] 2024-09-22T03:18:43.077 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T03:18:43.077 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:18:44.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-22T03:18:49.178 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T03:18:49.563 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~6s 2024-09-22T03:18:54.178 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 1, 3] 2024-09-22T03:18:54.179 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:18:54.179 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [2, 1, 3] 2024-09-22T03:18:54.179 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:18:54.179 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:18:54.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:18:54.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-22T03:19:00.280 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:19:00.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2024-09-22T03:19:05.281 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 1] live_osds: [2, 3] 2024-09-22T03:19:05.281 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:19:05.281 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:19:05.281 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:19:05.281 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:19:05.284 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:19:05.285 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:19:05.395 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:19:05.399 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:05.399 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:19:05.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:19:10.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:19:10.496 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:19:10.499 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:10.500 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:19:13.067 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:19:15.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:19:15.597 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:19:15.600 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:15.601 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:19:20.601 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:19:20.701 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:19:20.702 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [] 2024-09-22T03:19:20.702 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:19:23.653 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:19:23.654 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:19:25.654 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:19:25.711 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 3, 1] 2024-09-22T03:19:25.711 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:19:25.711 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-22T03:19:25.711 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:19:25.711 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:19:25.714 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:19:25.714 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:19:25.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:25.787+0000 7f56ff26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:25.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:25.793+0000 7f56ff26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:25.802 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:19:25.847 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:19:25.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:25.850 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:19:26.226 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:19:26.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:19:28.227 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:19:28.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:28.365+0000 7f5d7c197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:28.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:28.371+0000 7f5d7c197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:28.378 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:19:28.546 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:19:28.547 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:19:30.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:19:30.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:30.666+0000 7fb715423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:30.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:30.673+0000 7fb715423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:30.682 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:19:30.851 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:19:30.956 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:19:30.957 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:19:30.963 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:19:30.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:30.967 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:19:32.958 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:19:33.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:33.064+0000 7f7318f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:33.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:33.070+0000 7f7318f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:33.076 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:19:33.292 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:19:33.292 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:19:35.293 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:19:35.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:35.440+0000 7f7de6c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:35.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:35.445+0000 7f7de6c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:35.452 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:19:35.738 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:19:35.738 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:19:35.968 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:19:36.103 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:19:36.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:36.107 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:19:37.740 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:19:37.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:37.858+0000 7fdaa3a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:37.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:37.864+0000 7fdaa3a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:37.871 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:19:38.099 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:19:38.099 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:19:38.326 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Removed 22348 objects 2024-09-22T03:19:38.326 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Clean up completed and total clean up time :225.098 2024-09-22T03:19:39.417 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-22T03:19:39.417 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-22T03:19:39.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:39.570+0000 7f8191f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:39.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:39.583+0000 7f8191f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:40.060 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_1' removed 2024-09-22T03:19:40.075 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:19:40.075 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:19:40.076 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-22T03:19:40.076 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_2 16 2024-09-22T03:19:40.100 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:19:40.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:40.215+0000 7f369da58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:40.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:40.229+0000 7f369da58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:40.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:40.277+0000 7fda60d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:40.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:40.283+0000 7fda60d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:41.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:19:41.209 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:19:41.209 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [] 2024-09-22T03:19:41.209 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:19:41.399 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_2' created 2024-09-22T03:19:41.416 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-22T03:19:41.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:41.527+0000 7f5122e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:41.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:41.539+0000 7f5122e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:42.400 INFO:teuthology.orchestra.run.smithi146.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-22T03:19:42.417 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:19:42.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:42.579+0000 7f4b61597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:42.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:42.584+0000 7f4b61597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:43.419 DEBUG:teuthology.orchestra.run.smithi146:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-22T03:19:43.422 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:19:43.423 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:19:43.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:hints = 1 2024-09-22T03:19:43.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:19:43.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object prefix: benchmark_data_smithi146_42560 2024-09-22T03:19:43.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:19:43.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:19:44.485 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 1 16 414 398 24.8736 24.875 0.0228548 0.0392601 2024-09-22T03:19:44.652 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:19:44.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:44.795+0000 7ff77622c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:44.805 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:44.803+0000 7ff77622c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:45.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 2 16 805 789 24.6546 24.4375 0.0419018 0.0401618 2024-09-22T03:19:46.219 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 1, 0] 2024-09-22T03:19:46.219 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:19:46.219 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:19:46.220 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:19:46.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.355+0000 7fc514c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.366+0000 7fc514c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.390 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:19:46.391 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:19:46.394 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:46.395 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:19:46.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:19:46.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 3 16 1210 1194 24.8732 25.3125 0.0493953 0.0398624 2024-09-22T03:19:46.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.549+0000 7f775b8fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.561+0000 7f775b8fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:19:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:19:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:19:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:19:46.593 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:46.593 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:19:46.593 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:19:46.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.733+0000 7fa22d597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.747+0000 7fa22d597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:19:46.778 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:19:46.779 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:19:46.779 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:19:46.779 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:19:46.783 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:46.783 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:19:46.783 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:19:46.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.923+0000 7f29e442c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:46.936+0000 7f29e442c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:46.970 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:19:46.970 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:19:46.970 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:19:46.970 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:19:46.971 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:19:46.974 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:19:46.974 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:19:47.112 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:19:47.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:47.260+0000 7f1725ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:47.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:47.268+0000 7f1725ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:47.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 4 16 1608 1592 24.8731 24.875 0.0279078 0.0400081 2024-09-22T03:19:48.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 5 16 2023 2007 25.0855 25.9375 0.0374513 0.0396957 2024-09-22T03:19:49.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 6 16 2420 2404 25.0396 24.8125 0.047212 0.0397973 2024-09-22T03:19:49.550 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:19:49.700 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:49.696+0000 7f2e1106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:49.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:49.704+0000 7f2e1106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:50.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 7 16 2817 2801 25.0069 24.8125 0.0334227 0.0398604 2024-09-22T03:19:51.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 8 16 3214 3198 24.9823 24.8125 0.0234961 0.0399371 2024-09-22T03:19:51.936 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:19:51.975 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 1, 0] 2024-09-22T03:19:51.976 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:19:51.976 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [2, 3, 1, 0] 2024-09-22T03:19:51.976 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T03:19:51.976 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:19:52.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:52.059+0000 7f6681197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:52.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:19:52.063+0000 7f6681197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:19:52.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 9 16 3428 3412 23.6924 13.375 0.0161686 0.0398732 2024-09-22T03:19:53.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 10 16 3428 3412 21.3231 0 - 0.0398732 2024-09-22T03:19:54.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 11 16 3428 3412 19.3846 0 - 0.0398732 2024-09-22T03:19:55.486 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 12 16 3428 3412 17.7692 0 - 0.0398732 2024-09-22T03:19:55.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T03:19:56.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 13 16 3428 3412 16.4023 0 - 0.0398732 2024-09-22T03:19:57.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 14 16 3428 3412 15.2307 0 - 0.0398732 2024-09-22T03:19:58.077 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T03:19:58.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 15 16 3428 3412 14.2153 0 - 0.0398732 2024-09-22T03:19:59.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 16 16 3428 3412 13.3268 0 - 0.0398732 2024-09-22T03:20:00.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 17 16 3428 3412 12.5428 0 - 0.0398732 2024-09-22T03:20:01.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T03:20:01.487 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 18 16 3428 3412 11.846 0 - 0.0398732 2024-09-22T03:20:02.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 19 16 3428 3412 11.2225 0 - 0.0398732 2024-09-22T03:20:03.078 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [2, 1, 0] 2024-09-22T03:20:03.078 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:03.078 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-22T03:20:03.078 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:20:03.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:20:03.081 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:20:03.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:20:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:20:03.196 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:03.197 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:20:03.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:20:03.487117+0000 min lat: 0.00842921 max lat: 0.0897045 avg lat: 0.0398732 2024-09-22T03:20:03.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:20:03.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 20 16 3428 3412 10.6614 0 - 0.0398732 2024-09-22T03:20:04.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 21 16 3428 3412 10.1536 0 - 0.0398732 2024-09-22T03:20:05.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 22 16 3428 3412 9.6921 0 - 0.0398732 2024-09-22T03:20:06.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 23 16 3428 3412 9.2707 0 - 0.0398732 2024-09-22T03:20:07.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 24 16 3428 3412 8.88441 0 - 0.0398732 2024-09-22T03:20:08.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:20:08.297 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:20:08.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:08.300 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:20:08.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 25 16 3428 3412 8.52903 0 - 0.0398732 2024-09-22T03:20:09.488 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 26 16 3428 3412 8.20099 0 - 0.0398732 2024-09-22T03:20:10.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 27 16 3428 3412 7.89724 0 - 0.0398732 2024-09-22T03:20:11.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 28 16 3428 3412 7.6152 0 - 0.0398732 2024-09-22T03:20:12.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 29 16 3428 3412 7.3526 0 - 0.0398732 2024-09-22T03:20:13.072 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:20:13.078 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:20:13.079 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:20:13.301 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:20:13.407 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:20:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [] 2024-09-22T03:20:13.407 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:20:13.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 30 16 3428 3412 7.10752 0 - 0.0398732 2024-09-22T03:20:14.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 31 16 3428 3412 6.87824 0 - 0.0398732 2024-09-22T03:20:15.080 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:15.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:15.221+0000 7fc95f557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:15.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:15.227+0000 7fc95f557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:15.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 32 16 3521 3505 6.84492 0.252717 0.0448622 0.145988 2024-09-22T03:20:16.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 33 16 3929 3913 7.41014 25.5 0.0507869 0.134856 2024-09-22T03:20:17.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:17.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 34 16 4314 4298 7.89985 24.0625 0.065131 0.126472 2024-09-22T03:20:17.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:17.596+0000 7fdf11eb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:17.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:17.603+0000 7fdf11eb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:18.417 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:20:18.417 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:18.418 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:20:18.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 35 15 4711 4696 8.38478 24.875 0.0452692 0.119196 2024-09-22T03:20:18.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:18.569+0000 7f0453c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:18.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:18.580+0000 7f0453c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:18.838 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:20:18.838 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:20:18.858 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-22T03:20:17.342039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":220267059,"num_objects":3356,"num_object_clones":0,"num_object_copies":10068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3491,"num_write_kb":216424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3449,"ondisk_log_size":3449,"up":68,"acting":68,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":274888,"kb_used_data":42628,"kb_used_omap":0,"kb_used_meta":232192,"kb_avail":377212472,"statfs":{"total":386547056640,"available":386265571328,"internally_reserved":0,"allocated":43651072,"data_stored":643635974,"data_compressed":30500162,"data_compressed_allocated":40153088,"data_compressed_original":642203648,"omap_allocated":0,"internal_metadata":237764608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000902"},"pg_stats":[{"pgid":"5.f","version":"102'199","reported_seq":212,"reported_epoch":103,"state":"peering","last_fresh":"2024-09-22T03:19:52.037208+0000","last_change":"2024-09-22T03:20:13.079030+0000","last_active":"2024-09-22T03:20:13.079030+0000","last_peered":"2024-09-22T03:19:52.037208+0000","last_clean":"2024-09-22T03:19:52.037208+0000","last_became_active":"2024-09-22T03:19:42.479559+0000","last_became_peered":"2024-09-22T03:19:42.479559+0000","last_unstale":"2024-09-22T03:19:52.037208+0000","last_undegraded":"2024-09-22T03:19:52.037208+0000","last_fullsized":"2024-09-22T03:19:52.037208+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":199,"num_write_kb":12736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"102'213","reported_seq":228,"reported_epoch":104,"state":"peering","last_fresh":"2024-09-22T03:20:14.092103+0000","last_change":"2024-09-22T03:20:13.080488+0000","last_active":"2024-09-22T03:20:13.080488+0000","last_peered":"2024-09-22T03:19:52.010409+0000","last_clean":"2024-09-22T03:19:52.010409+0000","last_became_active":"2024-09-22T03:19:42.492830+0000","last_became_peered":"2024-09-22T03:19:42.492830+0000","last_unstale":"2024-09-22T03:20:14.092103+0000","last_undegraded":"2024-09-22T03:20:14.092103+0000","last_fullsized":"2024-09-22T03:20:14.092103+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":213,"num_object_clones":0,"num_object_copies":639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":213,"num_write_kb":13632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"102'227","reported_seq":242,"reported_epoch":104,"state":"peering","last_fresh":"2024-09-22T03:20:14.101162+0000","last_change":"2024-09-22T03:20:13.105600+0000","last_active":"2024-09-22T03:20:13.105600+0000","last_peered":"2024-09-22T03:19:52.024031+0000","last_clean":"2024-09-22T03:19:52.024031+0000","last_became_active":"2024-09-22T03:19:42.498600+0000","last_became_peered":"2024-09-22T03:19:42.498600+0000","last_unstale":"2024-09-22T03:20:14.101162+0000","last_undegraded":"2024-09-22T03:20:14.101162+0000","last_fullsized":"2024-09-22T03:20:14.101162+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14876672,"num_objects":227,"num_object_clones":0,"num_object_copies":681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":14528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"102'180","reported_seq":193,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-22T03:19:50.563143+0000","last_change":"2024-09-22T03:19:42.494939+0000","last_active":"2024-09-22T03:19:50.563143+0000","last_peered":"2024-09-22T03:19:50.563143+0000","last_clean":"2024-09-22T03:19:50.563143+0000","last_became_active":"2024-09-22T03:19:42.491161+0000","last_became_peered":"2024-09-22T03:19:42.491161+0000","last_unstale":"2024-09-22T03:20:13.069128+0000","last_undegraded":"2024-09-22T03:19:50.563143+0000","last_fullsized":"2024-09-22T03:19:50.563143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11862016,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":11584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"102'240","reported_seq":255,"reported_epoch":104,"state":"peering","last_fresh":"2024-09-22T03:20:14.098704+0000","last_change":"2024-09-22T03:20:13.100714+0000","last_active":"2024-09-22T03:20:13.100714+0000","last_peered":"2024-09-22T03:19:52.015259+0000","last_clean":"2024-09-22T03:19:52.015259+0000","last_became_active":"2024-09-22T03:19:42.499025+0000","last_became_peered":"2024-09-22T03:19:42.499025+0000","last_unstale":"2024-09-22T03:20:14.098704+0000","last_undegraded":"2024-09-22T03:20:14.098704+0000","last_fullsized":"2024-09-22T03:20:14.098704+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":240,"num_object_clones":0,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"102'199","reported_seq":212,"reported_epoch":103,"state":"peering","last_fresh":"2024-09-22T03:19:51.966041+0000","last_change":"2024-09-22T03:20:13.103178+0000","last_active":"2024-09-22T03:20:13.103178+0000","last_peered":"2024-09-22T03:19:51.966041+0000","last_clean":"2024-09-22T03:19:51.966041+0000","last_became_active":"2024-09-22T03:19:42.531292+0000","last_became_peered":"2024-09-22T03:19:42.531292+0000","last_unstale":"2024-09-22T03:19:51.966041+0000","last_undegraded":"2024-09-22T03:19:51.966041+0000","last_fullsized":"2024-09-22T03:19:51.966041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":199,"num_write_kb":12736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"102'217","reported_seq":239,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:15.100021+0000","last_change":"2024-09-22T03:19:42.473200+0000","last_active":"2024-09-22T03:20:15.100021+0000","last_peered":"2024-09-22T03:20:15.100021+0000","last_clean":"2024-09-22T03:20:15.100021+0000","last_became_active":"2024-09-22T03:19:42.468978+0000","last_became_peered":"2024-09-22T03:19:42.468978+0000","last_unstale":"2024-09-22T03:20:15.100021+0000","last_undegraded":"2024-09-22T03:20:15.100021+0000","last_fullsized":"2024-09-22T03:20:15.100021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":13952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"102'209","reported_seq":227,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:20:14.082185+0000","last_change":"2024-09-22T03:19:42.477912+0000","last_active":"2024-09-22T03:20:14.082185+0000","last_peered":"2024-09-22T03:20:14.082185+0000","last_clean":"2024-09-22T03:20:14.082185+0000","last_became_active":"2024-09-22T03:19:42.469937+0000","last_became_peered":"2024-09-22T03:19:42.469937+0000","last_unstale":"2024-09-22T03:20:14.082185+0000","last_undegraded":"2024-09-22T03:20:14.082185+0000","last_fullsized":"2024-09-22T03:20:14.082185+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":209,"num_write_kb":13376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"102'195","reported_seq":210,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-22T03:20:13.097554+0000","last_change":"2024-09-22T03:19:42.489336+0000","last_active":"2024-09-22T03:20:13.097554+0000","last_peered":"2024-09-22T03:20:13.097554+0000","last_clean":"2024-09-22T03:20:13.097554+0000","last_became_active":"2024-09-22T03:19:42.482335+0000","last_became_peered":"2024-09-22T03:19:42.482335+0000","last_unstale":"2024-09-22T03:20:13.097554+0000","last_undegraded":"2024-09-22T03:20:13.097554+0000","last_fullsized":"2024-09-22T03:20:13.097554+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":195,"num_object_clones":0,"num_object_copies":585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":195,"num_write_kb":12480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-22T03:19:43.420255+0000","last_change":"2024-09-22T03:19:40.089439+0000","last_active":"2024-09-22T03:19:43.420255+0000","last_peered":"2024-09-22T03:19:43.420255+0000","last_clean":"2024-09-22T03:19:43.420255+0000","last_became_active":"2024-09-22T03:19:39.463040+0000","last_became_peered":"2024-09-22T03:19:39.463040+0000","last_unstale":"2024-09-22T03:20:13.069131+0000","last_undegraded":"2024-09-22T03:19:43.420255+0000","last_fullsized":"2024-09-22T03:19:43.420255+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"102'234","reported_seq":247,"reported_epoch":103,"state":"peering","last_fresh":"2024-09-22T03:19:51.970241+0000","last_change":"2024-09-22T03:20:13.098751+0000","last_active":"2024-09-22T03:20:13.098751+0000","last_peered":"2024-09-22T03:19:51.970241+0000","last_clean":"2024-09-22T03:19:51.970241+0000","last_became_active":"2024-09-22T03:19:42.522226+0000","last_became_peered":"2024-09-22T03:19:42.522226+0000","last_unstale":"2024-09-22T03:19:51.970241+0000","last_undegraded":"2024-09-22T03:19:51.970241+0000","last_fullsized":"2024-09-22T03:19:51.970241+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15335424,"num_objects":234,"num_object_clones":0,"num_object_copies":702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":234,"num_write_kb":14976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":212,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:20:14.102849+0000","last_change":"2024-09-22T03:19:40.083887+0000","last_active":"2024-09-22T03:20:14.102849+0000","last_peered":"2024-09-22T03:20:14.102849+0000","last_clean":"2024-09-22T03:20:14.102849+0000","last_became_active":"2024-09-22T03:19:39.586255+0000","last_became_peered":"2024-09-22T03:19:39.586255+0000","last_unstale":"2024-09-22T03:20:14.102849+0000","last_undegraded":"2024-09-22T03:20:14.102849+0000","last_fullsized":"2024-09-22T03:20:14.102849+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"102'219","reported_seq":232,"reported_epoch":103,"state":"peering","last_fresh":"2024-09-22T03:19:52.042876+0000","last_change":"2024-09-22T03:20:13.083536+0000","last_active":"2024-09-22T03:20:13.083536+0000","last_peered":"2024-09-22T03:19:52.042876+0000","last_clean":"2024-09-22T03:19:52.042876+0000","last_became_active":"2024-09-22T03:19:42.462642+0000","last_became_peered":"2024-09-22T03:19:42.462642+0000","last_unstale":"2024-09-22T03:19:52.042876+0000","last_undegraded":"2024-09-22T03:19:52.042876+0000","last_fullsized":"2024-09-22T03:19:52.042876+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":219,"num_object_clones":0,"num_object_copies":657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":299,"reported_epoch":103,"state":"peering","last_fresh":"2024-09-22T03:19:43.423500+0000","last_change":"2024-09-22T03:20:13.088785+0000","last_active":"2024-09-22T03:20:13.088785+0000","last_peered":"2024-09-22T03:19:43.423500+0000","last_clean":"2024-09-22T03:19:43.423500+0000","last_became_active":"2024-09-22T03:19:22.110583+0000","last_became_peered":"2024-09-22T03:19:22.110583+0000","last_unstale":"2024-09-22T03:19:43.423500+0000","last_undegraded":"2024-09-22T03:19:43.423500+0000","last_fullsized":"2024-09-22T03:19:43.423500+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"102'199","reported_seq":217,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:20:14.102932+0000","last_change":"2024-09-22T03:19:42.520982+0000","last_active":"2024-09-22T03:20:14.102932+0000","last_peered":"2024-09-22T03:20:14.102932+0000","last_clean":"2024-09-22T03:20:14.102932+0000","last_became_active":"2024-09-22T03:19:42.518086+0000","last_became_peered":"2024-09-22T03:19:42.518086+0000","last_unstale":"2024-09-22T03:20:14.102932+0000","last_undegraded":"2024-09-22T03:20:14.102932+0000","last_fullsized":"2024-09-22T03:20:14.102932+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":199,"num_write_kb":12736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":171,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-22T03:19:43.420564+0000","last_change":"2024-09-22T03:19:40.119302+0000","last_active":"2024-09-22T03:19:43.420564+0000","last_peered":"2024-09-22T03:19:43.420564+0000","last_clean":"2024-09-22T03:19:43.420564+0000","last_became_active":"2024-09-22T03:19:39.465874+0000","last_became_peered":"2024-09-22T03:19:39.465874+0000","last_unstale":"2024-09-22T03:20:13.069133+0000","last_undegraded":"2024-09-22T03:19:43.420564+0000","last_fullsized":"2024-09-22T03:19:43.420564+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":335,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:20:14.103220+0000","last_change":"2024-09-22T03:19:39.588674+0000","last_active":"2024-09-22T03:20:14.103220+0000","last_peered":"2024-09-22T03:20:14.103220+0000","last_clean":"2024-09-22T03:20:14.103220+0000","last_became_active":"2024-09-22T03:19:39.528532+0000","last_became_peered":"2024-09-22T03:19:39.528532+0000","last_unstale":"2024-09-22T03:20:14.103220+0000","last_undegraded":"2024-09-22T03:20:14.103220+0000","last_fullsized":"2024-09-22T03:20:14.103220+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"102'243","reported_seq":261,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:20:14.094529+0000","last_change":"2024-09-22T03:19:42.501027+0000","last_active":"2024-09-22T03:20:14.094529+0000","last_peered":"2024-09-22T03:20:14.094529+0000","last_clean":"2024-09-22T03:20:14.094529+0000","last_became_active":"2024-09-22T03:19:42.494283+0000","last_became_peered":"2024-09-22T03:19:42.494283+0000","last_unstale":"2024-09-22T03:20:14.094529+0000","last_undegraded":"2024-09-22T03:20:14.094529+0000","last_fullsized":"2024-09-22T03:20:14.094529+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":243,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:15.094997+0000","last_change":"2024-09-22T03:19:40.106505+0000","last_active":"2024-09-22T03:20:15.094997+0000","last_peered":"2024-09-22T03:20:15.094997+0000","last_clean":"2024-09-22T03:20:15.094997+0000","last_became_active":"2024-09-22T03:19:39.487648+0000","last_became_peered":"2024-09-22T03:19:39.487648+0000","last_unstale":"2024-09-22T03:20:15.094997+0000","last_undegraded":"2024-09-22T03:20:15.094997+0000","last_fullsized":"2024-09-22T03:20:15.094997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"102'190","reported_seq":203,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-22T03:19:50.537810+0000","last_change":"2024-09-22T03:19:42.471335+0000","last_active":"2024-09-22T03:19:50.537810+0000","last_peered":"2024-09-22T03:19:50.537810+0000","last_clean":"2024-09-22T03:19:50.537810+0000","last_became_active":"2024-09-22T03:19:42.465267+0000","last_became_peered":"2024-09-22T03:19:42.465267+0000","last_unstale":"2024-09-22T03:20:13.069136+0000","last_undegraded":"2024-09-22T03:19:50.537810+0000","last_fullsized":"2024-09-22T03:19:50.537810+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12517376,"num_objects":191,"num_object_clones":0,"num_object_copies":573,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":12224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":285,"reported_epoch":103,"state":"peering","last_fresh":"2024-09-22T03:19:43.427016+0000","last_change":"2024-09-22T03:20:13.106800+0000","last_active":"2024-09-22T03:20:13.106800+0000","last_peered":"2024-09-22T03:19:43.427016+0000","last_clean":"2024-09-22T03:19:43.427016+0000","last_became_active":"2024-09-22T03:19:39.503060+0000","last_became_peered":"2024-09-22T03:19:39.503060+0000","last_unstale":"2024-09-22T03:19:43.427016+0000","last_undegraded":"2024-09-22T03:19:43.427016+0000","last_fullsized":"2024-09-22T03:19:43.427016+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"102'184","reported_seq":198,"reported_epoch":104,"state":"peering","last_fresh":"2024-09-22T03:20:13.092496+0000","last_change":"2024-09-22T03:20:13.092496+0000","last_active":"2024-09-22T03:20:13.092496+0000","last_peered":"2024-09-22T03:19:51.902726+0000","last_clean":"2024-09-22T03:19:51.902726+0000","last_became_active":"2024-09-22T03:19:42.465890+0000","last_became_peered":"2024-09-22T03:19:42.465890+0000","last_unstale":"2024-09-22T03:20:13.092496+0000","last_undegraded":"2024-09-22T03:20:13.092496+0000","last_fullsized":"2024-09-22T03:20:13.092496+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058624,"num_objects":184,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":11776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":220,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:15.170613+0000","last_change":"2024-09-22T03:20:15.171530+0000","last_active":"2024-09-22T03:20:15.170613+0000","last_peered":"2024-09-22T03:20:15.170613+0000","last_clean":"2024-09-22T03:19:43.427451+0000","last_became_active":"2024-09-22T03:20:15.167293+0000","last_became_peered":"2024-09-22T03:20:15.167293+0000","last_unstale":"2024-09-22T03:20:15.170613+0000","last_undegraded":"2024-09-22T03:20:15.170613+0000","last_fullsized":"2024-09-22T03:20:15.170613+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"102'201","reported_seq":214,"reported_epoch":103,"state":"peering","last_fresh":"2024-09-22T03:19:52.018862+0000","last_change":"2024-09-22T03:20:13.108167+0000","last_active":"2024-09-22T03:20:13.108167+0000","last_peered":"2024-09-22T03:19:52.018862+0000","last_clean":"2024-09-22T03:19:52.018862+0000","last_became_active":"2024-09-22T03:19:42.506880+0000","last_became_peered":"2024-09-22T03:19:42.506880+0000","last_unstale":"2024-09-22T03:19:52.018862+0000","last_undegraded":"2024-09-22T03:19:52.018862+0000","last_fullsized":"2024-09-22T03:19:52.018862+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":201,"num_object_clones":0,"num_object_copies":603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":201,"num_write_kb":12864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":198,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:15.153837+0000","last_change":"2024-09-22T03:20:15.154936+0000","last_active":"2024-09-22T03:20:15.153837+0000","last_peered":"2024-09-22T03:20:15.153837+0000","last_clean":"2024-09-22T03:19:43.423081+0000","last_became_active":"2024-09-22T03:20:15.150206+0000","last_became_peered":"2024-09-22T03:20:15.150206+0000","last_unstale":"2024-09-22T03:20:15.153837+0000","last_undegraded":"2024-09-22T03:20:15.153837+0000","last_fullsized":"2024-09-22T03:20:15.153837+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":219676672,"num_objects":3352,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3352,"num_write_kb":214528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40546304,"data_stored":648740864,"data_compressed":30802358,"data_compressed_allocated":40546304,"data_compressed_original":648740864,"omap_allocated":0,"internal_metadata":0},"log_size":3349,"ondisk_log_size":3349,"up":43,"acting":43,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":22,"acting":22,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":81,"seq":347892351061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65564,"kb_used_data":8264,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94306276,"statfs":{"total":96636764160,"available":96569626624,"internally_reserved":0,"allocated":8462336,"data_stored":122107668,"data_compressed":5781331,"data_compressed_allocated":7610368,"data_compressed_original":121765888,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102196,"kb_used_data":12900,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94269644,"statfs":{"total":96636764160,"available":96532115456,"internally_reserved":0,"allocated":13209600,"data_stored":197671220,"data_compressed":9371212,"data_compressed_allocated":12337152,"data_compressed_original":197312512,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":95,"seq":408021893174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48672,"kb_used_data":9488,"kb_used_omap":0,"kb_used_meta":39168,"kb_avail":94323168,"statfs":{"total":96636764160,"available":96586924032,"internally_reserved":0,"allocated":9715712,"data_stored":141386607,"data_compressed":6698289,"data_compressed_allocated":8818688,"data_compressed_original":141017088,"omap_allocated":0,"internal_metadata":40108032},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58456,"kb_used_data":11976,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94313384,"statfs":{"total":96636764160,"available":96576905216,"internally_reserved":0,"allocated":12263424,"data_stored":182470479,"data_compressed":8649330,"data_compressed_allocated":11386880,"data_compressed_original":182108160,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11345920,"data_stored":181534720,"data_compressed":8619358,"data_compressed_allocated":11345920,"data_compressed_original":181534720,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8966144,"data_stored":143458304,"data_compressed":6811469,"data_compressed_allocated":8966144,"data_compressed_original":143458304,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12296192,"data_stored":196739072,"data_compressed":9341240,"data_compressed_allocated":12296192,"data_compressed_original":196739072,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7938048,"data_stored":127008768,"data_compressed":6030291,"data_compressed_allocated":7938048,"data_compressed_original":127008768,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:20:19.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 36 16 5110 5094 8.84277 24.875 0.0532101 0.113015 2024-09-22T03:20:19.867 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:20.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:20.012+0000 7f5197997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:20.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:20.020+0000 7f5197997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:20.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 37 16 5492 5476 9.24898 23.875 0.0669962 0.10805 2024-09-22T03:20:21.489 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 38 16 5879 5863 9.64204 24.1875 0.0437063 0.10364 2024-09-22T03:20:22.340 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:22.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 39 16 6275 6259 10.0294 24.75 0.0358852 0.0996153 2024-09-22T03:20:22.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:22.476+0000 7f063c357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:22.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:22.558+0000 7f063c357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:23.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:20:23.489270+0000 min lat: 0.00842921 max lat: 23.3056 avg lat: 0.0970351 2024-09-22T03:20:23.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:20:23.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 40 16 6607 6591 10.2973 20.75 0.0148046 0.0970351 2024-09-22T03:20:23.860 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:20:23.860 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:23.860 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:20:24.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:24.001+0000 7f34a246c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:24.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:24.012+0000 7f34a246c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:24.341 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:20:24.341 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:20:24.362 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-22T03:20:23.343181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":381289011,"num_objects":5813,"num_object_clones":0,"num_object_copies":17439,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5948,"num_write_kb":373672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5893,"ondisk_log_size":5893,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":295988,"kb_used_data":72180,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":377191372,"statfs":{"total":386547056640,"available":386243964928,"internally_reserved":0,"allocated":73912320,"data_stored":1126412748,"data_compressed":53423154,"data_compressed_allocated":70324224,"data_compressed_original":1124941824,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":161087488,"num_objects":2458,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2458,"num_write_kb":157312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.263232"},"pg_stats":[{"pgid":"5.f","version":"106'390","reported_seq":425,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:23.024376+0000","last_change":"2024-09-22T03:20:15.340645+0000","last_active":"2024-09-22T03:20:23.024376+0000","last_peered":"2024-09-22T03:20:23.024376+0000","last_clean":"2024-09-22T03:20:23.024376+0000","last_became_active":"2024-09-22T03:20:15.157196+0000","last_became_peered":"2024-09-22T03:20:15.157196+0000","last_unstale":"2024-09-22T03:20:23.024376+0000","last_undegraded":"2024-09-22T03:20:23.024376+0000","last_fullsized":"2024-09-22T03:20:23.024376+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":391,"num_object_clones":0,"num_object_copies":1173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":391,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"105'322","reported_seq":352,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:19.169171+0000","last_change":"2024-09-22T03:20:15.256117+0000","last_active":"2024-09-22T03:20:19.169171+0000","last_peered":"2024-09-22T03:20:19.169171+0000","last_clean":"2024-09-22T03:20:19.169171+0000","last_became_active":"2024-09-22T03:20:15.158988+0000","last_became_peered":"2024-09-22T03:20:15.158988+0000","last_unstale":"2024-09-22T03:20:19.169171+0000","last_undegraded":"2024-09-22T03:20:19.169171+0000","last_fullsized":"2024-09-22T03:20:19.169171+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":323,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"105'325","reported_seq":353,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:19.185512+0000","last_change":"2024-09-22T03:20:15.286545+0000","last_active":"2024-09-22T03:20:19.185512+0000","last_peered":"2024-09-22T03:20:19.185512+0000","last_clean":"2024-09-22T03:20:19.185512+0000","last_became_active":"2024-09-22T03:20:15.178048+0000","last_became_peered":"2024-09-22T03:20:15.178048+0000","last_unstale":"2024-09-22T03:20:19.185512+0000","last_undegraded":"2024-09-22T03:20:19.185512+0000","last_fullsized":"2024-09-22T03:20:19.185512+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":326,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"106'422","reported_seq":447,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:23.025614+0000","last_change":"2024-09-22T03:20:15.171913+0000","last_active":"2024-09-22T03:20:23.025614+0000","last_peered":"2024-09-22T03:20:23.025614+0000","last_clean":"2024-09-22T03:20:23.025614+0000","last_became_active":"2024-09-22T03:20:15.164720+0000","last_became_peered":"2024-09-22T03:20:15.164720+0000","last_unstale":"2024-09-22T03:20:23.025614+0000","last_undegraded":"2024-09-22T03:20:23.025614+0000","last_fullsized":"2024-09-22T03:20:23.025614+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":423,"num_object_clones":0,"num_object_copies":1269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"105'325","reported_seq":357,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:19.192231+0000","last_change":"2024-09-22T03:20:15.292671+0000","last_active":"2024-09-22T03:20:19.192231+0000","last_peered":"2024-09-22T03:20:19.192231+0000","last_clean":"2024-09-22T03:20:19.192231+0000","last_became_active":"2024-09-22T03:20:15.171897+0000","last_became_peered":"2024-09-22T03:20:15.171897+0000","last_unstale":"2024-09-22T03:20:19.192231+0000","last_undegraded":"2024-09-22T03:20:19.192231+0000","last_fullsized":"2024-09-22T03:20:19.192231+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":326,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"106'384","reported_seq":410,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:23.288611+0000","last_change":"2024-09-22T03:20:15.174790+0000","last_active":"2024-09-22T03:20:23.288611+0000","last_peered":"2024-09-22T03:20:23.288611+0000","last_clean":"2024-09-22T03:20:23.288611+0000","last_became_active":"2024-09-22T03:20:15.164887+0000","last_became_peered":"2024-09-22T03:20:15.164887+0000","last_unstale":"2024-09-22T03:20:23.288611+0000","last_undegraded":"2024-09-22T03:20:23.288611+0000","last_fullsized":"2024-09-22T03:20:23.288611+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25231360,"num_objects":385,"num_object_clones":0,"num_object_copies":1155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":385,"num_write_kb":24640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"106'351","reported_seq":376,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.366316+0000","last_change":"2024-09-22T03:19:42.473200+0000","last_active":"2024-09-22T03:20:20.366316+0000","last_peered":"2024-09-22T03:20:20.366316+0000","last_clean":"2024-09-22T03:20:20.366316+0000","last_became_active":"2024-09-22T03:19:42.468978+0000","last_became_peered":"2024-09-22T03:19:42.468978+0000","last_unstale":"2024-09-22T03:20:20.366316+0000","last_undegraded":"2024-09-22T03:20:20.366316+0000","last_fullsized":"2024-09-22T03:20:20.366316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":352,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"105'331","reported_seq":353,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:19.114124+0000","last_change":"2024-09-22T03:19:42.477912+0000","last_active":"2024-09-22T03:20:19.114124+0000","last_peered":"2024-09-22T03:20:19.114124+0000","last_clean":"2024-09-22T03:20:19.114124+0000","last_became_active":"2024-09-22T03:19:42.469937+0000","last_became_peered":"2024-09-22T03:19:42.469937+0000","last_unstale":"2024-09-22T03:20:19.114124+0000","last_undegraded":"2024-09-22T03:20:19.114124+0000","last_fullsized":"2024-09-22T03:20:19.114124+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757952,"num_objects":332,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":332,"num_write_kb":21248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"106'386","reported_seq":411,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:22.821133+0000","last_change":"2024-09-22T03:19:42.489336+0000","last_active":"2024-09-22T03:20:22.821133+0000","last_peered":"2024-09-22T03:20:22.821133+0000","last_clean":"2024-09-22T03:20:22.821133+0000","last_became_active":"2024-09-22T03:19:42.482335+0000","last_became_peered":"2024-09-22T03:19:42.482335+0000","last_unstale":"2024-09-22T03:20:22.821133+0000","last_undegraded":"2024-09-22T03:20:22.821133+0000","last_fullsized":"2024-09-22T03:20:22.821133+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":387,"num_object_clones":0,"num_object_copies":1161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":387,"num_write_kb":24768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.085997+0000","last_change":"2024-09-22T03:20:20.093423+0000","last_active":"2024-09-22T03:20:20.085997+0000","last_peered":"2024-09-22T03:20:20.085997+0000","last_clean":"2024-09-22T03:20:20.085997+0000","last_became_active":"2024-09-22T03:20:15.171717+0000","last_became_peered":"2024-09-22T03:20:15.171717+0000","last_unstale":"2024-09-22T03:20:20.085997+0000","last_undegraded":"2024-09-22T03:20:20.085997+0000","last_fullsized":"2024-09-22T03:20:20.085997+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"106'433","reported_seq":464,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:23.219624+0000","last_change":"2024-09-22T03:20:15.325469+0000","last_active":"2024-09-22T03:20:23.219624+0000","last_peered":"2024-09-22T03:20:23.219624+0000","last_clean":"2024-09-22T03:20:23.219624+0000","last_became_active":"2024-09-22T03:20:15.188302+0000","last_became_peered":"2024-09-22T03:20:15.188302+0000","last_unstale":"2024-09-22T03:20:23.219624+0000","last_undegraded":"2024-09-22T03:20:23.219624+0000","last_fullsized":"2024-09-22T03:20:23.219624+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":434,"num_write_kb":27776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":215,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:15.124912+0000","last_change":"2024-09-22T03:19:40.083887+0000","last_active":"2024-09-22T03:20:15.124912+0000","last_peered":"2024-09-22T03:20:15.124912+0000","last_clean":"2024-09-22T03:20:15.124912+0000","last_became_active":"2024-09-22T03:19:39.586255+0000","last_became_peered":"2024-09-22T03:19:39.586255+0000","last_unstale":"2024-09-22T03:20:15.124912+0000","last_undegraded":"2024-09-22T03:20:15.124912+0000","last_fullsized":"2024-09-22T03:20:15.124912+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"106'424","reported_seq":457,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:23.109689+0000","last_change":"2024-09-22T03:20:15.310737+0000","last_active":"2024-09-22T03:20:23.109689+0000","last_peered":"2024-09-22T03:20:23.109689+0000","last_clean":"2024-09-22T03:20:23.109689+0000","last_became_active":"2024-09-22T03:20:15.170038+0000","last_became_peered":"2024-09-22T03:20:15.170038+0000","last_unstale":"2024-09-22T03:20:23.109689+0000","last_undegraded":"2024-09-22T03:20:23.109689+0000","last_fullsized":"2024-09-22T03:20:23.109689+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27852800,"num_objects":425,"num_object_clones":0,"num_object_copies":1275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":425,"num_write_kb":27200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":312,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.097504+0000","last_change":"2024-09-22T03:20:20.116543+0000","last_active":"2024-09-22T03:20:20.097504+0000","last_peered":"2024-09-22T03:20:20.097504+0000","last_clean":"2024-09-22T03:20:20.097504+0000","last_became_active":"2024-09-22T03:20:15.147909+0000","last_became_peered":"2024-09-22T03:20:15.147909+0000","last_unstale":"2024-09-22T03:20:20.097504+0000","last_undegraded":"2024-09-22T03:20:20.097504+0000","last_fullsized":"2024-09-22T03:20:20.097504+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"105'303","reported_seq":325,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:19.174758+0000","last_change":"2024-09-22T03:19:42.520982+0000","last_active":"2024-09-22T03:20:19.174758+0000","last_peered":"2024-09-22T03:20:19.174758+0000","last_clean":"2024-09-22T03:20:19.174758+0000","last_became_active":"2024-09-22T03:19:42.518086+0000","last_became_peered":"2024-09-22T03:19:42.518086+0000","last_unstale":"2024-09-22T03:20:19.174758+0000","last_undegraded":"2024-09-22T03:20:19.174758+0000","last_fullsized":"2024-09-22T03:20:19.174758+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19922944,"num_objects":304,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":171,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.087763+0000","last_change":"2024-09-22T03:20:20.093627+0000","last_active":"2024-09-22T03:20:20.087763+0000","last_peered":"2024-09-22T03:20:20.087763+0000","last_clean":"2024-09-22T03:20:20.087763+0000","last_became_active":"2024-09-22T03:20:15.159573+0000","last_became_peered":"2024-09-22T03:20:15.159573+0000","last_unstale":"2024-09-22T03:20:20.087763+0000","last_undegraded":"2024-09-22T03:20:20.087763+0000","last_fullsized":"2024-09-22T03:20:20.087763+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":338,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:15.124107+0000","last_change":"2024-09-22T03:19:39.588674+0000","last_active":"2024-09-22T03:20:15.124107+0000","last_peered":"2024-09-22T03:20:15.124107+0000","last_clean":"2024-09-22T03:20:15.124107+0000","last_became_active":"2024-09-22T03:19:39.528532+0000","last_became_peered":"2024-09-22T03:19:39.528532+0000","last_unstale":"2024-09-22T03:20:15.124107+0000","last_undegraded":"2024-09-22T03:20:15.124107+0000","last_fullsized":"2024-09-22T03:20:15.124107+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"105'326","reported_seq":348,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:19.151262+0000","last_change":"2024-09-22T03:19:42.501027+0000","last_active":"2024-09-22T03:20:19.151262+0000","last_peered":"2024-09-22T03:20:19.151262+0000","last_clean":"2024-09-22T03:20:19.151262+0000","last_became_active":"2024-09-22T03:19:42.494283+0000","last_became_peered":"2024-09-22T03:19:42.494283+0000","last_unstale":"2024-09-22T03:20:19.151262+0000","last_undegraded":"2024-09-22T03:20:19.151262+0000","last_fullsized":"2024-09-22T03:20:19.151262+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.075953+0000","last_change":"2024-09-22T03:19:40.106505+0000","last_active":"2024-09-22T03:20:20.075953+0000","last_peered":"2024-09-22T03:20:20.075953+0000","last_clean":"2024-09-22T03:20:20.075953+0000","last_became_active":"2024-09-22T03:19:39.487648+0000","last_became_peered":"2024-09-22T03:19:39.487648+0000","last_unstale":"2024-09-22T03:20:20.075953+0000","last_undegraded":"2024-09-22T03:20:20.075953+0000","last_fullsized":"2024-09-22T03:20:20.075953+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"106'422","reported_seq":447,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:23.106061+0000","last_change":"2024-09-22T03:20:15.209720+0000","last_active":"2024-09-22T03:20:23.106061+0000","last_peered":"2024-09-22T03:20:23.106061+0000","last_clean":"2024-09-22T03:20:23.106061+0000","last_became_active":"2024-09-22T03:20:15.198961+0000","last_became_peered":"2024-09-22T03:20:15.198961+0000","last_unstale":"2024-09-22T03:20:23.106061+0000","last_undegraded":"2024-09-22T03:20:23.106061+0000","last_fullsized":"2024-09-22T03:20:23.106061+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":423,"num_object_clones":0,"num_object_copies":1269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":298,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.103917+0000","last_change":"2024-09-22T03:20:20.111817+0000","last_active":"2024-09-22T03:20:20.103917+0000","last_peered":"2024-09-22T03:20:20.103917+0000","last_clean":"2024-09-22T03:20:20.103917+0000","last_became_active":"2024-09-22T03:20:15.168250+0000","last_became_peered":"2024-09-22T03:20:15.168250+0000","last_unstale":"2024-09-22T03:20:20.103917+0000","last_undegraded":"2024-09-22T03:20:20.103917+0000","last_fullsized":"2024-09-22T03:20:20.103917+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"105'251","reported_seq":274,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T03:20:19.110788+0000","last_change":"2024-09-22T03:20:15.149437+0000","last_active":"2024-09-22T03:20:19.110788+0000","last_peered":"2024-09-22T03:20:19.110788+0000","last_clean":"2024-09-22T03:20:19.110788+0000","last_became_active":"2024-09-22T03:20:15.144457+0000","last_became_peered":"2024-09-22T03:20:15.144457+0000","last_unstale":"2024-09-22T03:20:19.110788+0000","last_undegraded":"2024-09-22T03:20:19.110788+0000","last_fullsized":"2024-09-22T03:20:19.110788+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16515072,"num_objects":252,"num_object_clones":0,"num_object_copies":756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":224,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.076259+0000","last_change":"2024-09-22T03:20:20.082089+0000","last_active":"2024-09-22T03:20:20.076259+0000","last_peered":"2024-09-22T03:20:20.076259+0000","last_clean":"2024-09-22T03:20:20.076259+0000","last_became_active":"2024-09-22T03:20:15.167293+0000","last_became_peered":"2024-09-22T03:20:15.167293+0000","last_unstale":"2024-09-22T03:20:20.076259+0000","last_undegraded":"2024-09-22T03:20:20.076259+0000","last_fullsized":"2024-09-22T03:20:20.076259+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"106'398","reported_seq":429,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:23.184747+0000","last_change":"2024-09-22T03:20:15.266705+0000","last_active":"2024-09-22T03:20:23.184747+0000","last_peered":"2024-09-22T03:20:23.184747+0000","last_clean":"2024-09-22T03:20:23.184747+0000","last_became_active":"2024-09-22T03:20:15.171141+0000","last_became_peered":"2024-09-22T03:20:15.171141+0000","last_unstale":"2024-09-22T03:20:23.184747+0000","last_undegraded":"2024-09-22T03:20:23.184747+0000","last_fullsized":"2024-09-22T03:20:23.184747+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26148864,"num_objects":399,"num_object_clones":0,"num_object_copies":1197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":399,"num_write_kb":25536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":202,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.074349+0000","last_change":"2024-09-22T03:20:20.080550+0000","last_active":"2024-09-22T03:20:20.074349+0000","last_peered":"2024-09-22T03:20:20.074349+0000","last_clean":"2024-09-22T03:20:20.074349+0000","last_became_active":"2024-09-22T03:20:15.150206+0000","last_became_peered":"2024-09-22T03:20:15.150206+0000","last_unstale":"2024-09-22T03:20:20.074349+0000","last_undegraded":"2024-09-22T03:20:20.074349+0000","last_fullsized":"2024-09-22T03:20:20.074349+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":380698624,"num_objects":5809,"num_object_clones":0,"num_object_copies":17427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5809,"num_write_kb":371776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72429568,"data_stored":1158873088,"data_compressed":55026166,"data_compressed_allocated":72429568,"data_compressed_original":1158873088,"omap_allocated":0,"internal_metadata":0},"log_size":5793,"ondisk_log_size":5793,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676598793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63272,"kb_used_data":18584,"kb_used_omap":0,"kb_used_meta":44672,"kb_avail":94308568,"statfs":{"total":96636764160,"available":96571973632,"internally_reserved":0,"allocated":19030016,"data_stored":290680336,"data_compressed":13785395,"data_compressed_allocated":18145280,"data_compressed_original":290324480,"omap_allocated":0,"internal_metadata":45744128},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108444,"kb_used_data":23308,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94263396,"statfs":{"total":96636764160,"available":96525717504,"internally_reserved":0,"allocated":23867392,"data_stored":367685680,"data_compressed":17443740,"data_compressed_allocated":22962176,"data_compressed_original":367312896,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":95,"seq":408021893179,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57556,"kb_used_data":14212,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94314284,"statfs":{"total":96636764160,"available":96577826816,"internally_reserved":0,"allocated":14553088,"data_stored":218656816,"data_compressed":10367337,"data_compressed_allocated":13647872,"data_compressed_original":218284032,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66716,"kb_used_data":16076,"kb_used_omap":0,"kb_used_meta":50624,"kb_avail":94305124,"statfs":{"total":96636764160,"available":96568446976,"internally_reserved":0,"allocated":16461824,"data_stored":249389916,"data_compressed":11826682,"data_compressed_allocated":15568896,"data_compressed_original":249020416,"omap_allocated":0,"internal_metadata":51838976},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16560128,"data_stored":264962048,"data_compressed":12580934,"data_compressed_allocated":16560128,"data_compressed_original":264962048,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13930496,"data_stored":222887936,"data_compressed":10583213,"data_compressed_allocated":13930496,"data_compressed_original":222887936,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23736320,"data_stored":379781120,"data_compressed":18033056,"data_compressed_allocated":23736320,"data_compressed_original":379781120,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18202624,"data_stored":291241984,"data_compressed":13828963,"data_compressed_allocated":18202624,"data_compressed_original":291241984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:20:24.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 41 16 6989 6973 10.6284 23.875 0.0321499 0.0940415 2024-09-22T03:20:24.789 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:24.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:24.935+0000 7fd49f86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:24.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:24.941+0000 7fd49f86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:25.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 42 16 7360 7344 10.9274 23.1875 0.0304649 0.0914623 2024-09-22T03:20:26.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 43 16 7739 7723 11.2241 23.6875 0.0508488 0.0890514 2024-09-22T03:20:27.123 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:27.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:27.270+0000 7f50bbf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:27.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:27.277+0000 7f50bbf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:27.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 44 16 8125 8109 11.5173 24.125 0.0525297 0.086775 2024-09-22T03:20:28.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 45 16 8520 8504 11.8099 24.6875 0.0430726 0.0846365 2024-09-22T03:20:29.365 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:20:29.365 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:29.365 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:20:29.365 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:20:29.472 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:29.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 46 16 8897 8881 12.0653 23.5625 0.0354148 0.0828396 2024-09-22T03:20:29.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:29.514+0000 7f3e9bb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:29.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:29.529+0000 7f3e9bb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:29.556 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:29.564 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:29.564 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:29.564 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:20:29.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:29.622+0000 7fa976218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:29.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:29.628+0000 7fa976218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:29.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:29.759+0000 7f983166c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:29.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:29.773+0000 7f983166c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:29.798 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:29.798 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:29.798 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:29.798 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:29.799 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:29.799 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:29.799 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:29.799 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:29.799 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:29.799 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:29.799 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:29.800 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:29.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:29.803 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:29.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:20:29.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:29.989+0000 7fd1cdc23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:30.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:30.001+0000 7fd1cdc23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:30.040 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:30.041 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:30.041 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:30.041 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:30.044 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:30.044 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:30.044 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:20:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:30.189+0000 7ff711d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:30.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:30.200+0000 7ff711d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:30.243 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:30.244 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:30.245 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:30.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:30.247 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:30.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 47 16 9245 9229 12.2713 21.75 0.0572495 0.081443 2024-09-22T03:20:31.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 48 16 9609 9593 12.4896 22.75 0.0347636 0.0800249 2024-09-22T03:20:31.774 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:31.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:31.912+0000 7ff1afa2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:31.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:31.920+0000 7ff1afa2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:32.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 49 16 9976 9960 12.7028 22.9375 0.0881581 0.078671 2024-09-22T03:20:33.490 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 50 16 10368 10352 12.9387 24.5 0.0628396 0.0772421 2024-09-22T03:20:34.254 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:34.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:34.399+0000 7f39a3317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:34.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:34.407+0000 7f39a3317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:34.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 51 16 10733 10717 13.1322 22.8125 0.0508648 0.0760944 2024-09-22T03:20:35.248 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:20:35.248 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:35.248 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:20:35.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:20:35.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:35.396+0000 7fee4e8ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:35.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:35.408+0000 7fee4e8ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:35.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 52 16 11097 11081 13.3172 22.75 0.0462873 0.0750552 2024-09-22T03:20:35.500 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:35.500 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:35.500 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:35.500 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:35.501 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:35.504 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:35.504 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:35.504 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:20:35.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:35.646+0000 7face6d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:35.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:35.658+0000 7face6d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:35.700 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:35.704 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:35.704 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:35.704 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:20:35.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:35.839+0000 7f274aa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:35.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:35.855+0000 7f274aa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:35.878 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:35.879 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:35.883 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:35.884 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:35.884 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:20:36.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:36.019+0000 7fd2f0b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:36.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:36.031+0000 7fd2f0b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:36.084 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:36.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:36.088 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:36.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 53 16 11450 11434 13.4821 22.0625 0.0563679 0.0741384 2024-09-22T03:20:36.727 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:36.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:36.874+0000 7f4ba642c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:36.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:36.881+0000 7f4ba642c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:37.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 54 16 11803 11787 13.641 22.0625 0.0328789 0.0732792 2024-09-22T03:20:38.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 55 16 12175 12159 13.8157 23.25 0.0723294 0.072345 2024-09-22T03:20:39.061 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:39.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:39.192+0000 7f30d8557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:39.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:39.197+0000 7f30d8557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:39.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 56 16 12518 12502 13.9517 21.4375 0.0552068 0.0716427 2024-09-22T03:20:40.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 57 16 12879 12863 14.1028 22.5625 0.0411054 0.0708733 2024-09-22T03:20:41.088 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:20:41.088 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:41.088 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:20:41.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:41.228+0000 7f7b85b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:41.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:41.237+0000 7f7b85b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:41.485 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:41.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 58 16 13227 13211 14.2346 21.75 0.0230354 0.070221 2024-09-22T03:20:41.510 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:20:41.510 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:20:41.534 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-22T03:20:41.346724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":806027827,"num_objects":12294,"num_object_clones":0,"num_object_copies":36882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12429,"num_write_kb":788456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12374,"ondisk_log_size":12374,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":379012,"kb_used_data":149632,"kb_used_omap":0,"kb_used_meta":229312,"kb_avail":377108348,"statfs":{"total":386547056640,"available":386158948352,"internally_reserved":0,"allocated":153223168,"data_stored":2395258509,"data_compressed":113674583,"data_compressed_allocated":149626880,"data_compressed_original":2393784320,"omap_allocated":0,"internal_metadata":234815488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":246218752,"num_objects":3757,"num_object_clones":0,"num_object_copies":11271,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3757,"num_write_kb":240448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002425"},"pg_stats":[{"pgid":"5.f","version":"106'735","reported_seq":770,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.254223+0000","last_change":"2024-09-22T03:20:15.340645+0000","last_active":"2024-09-22T03:20:38.254223+0000","last_peered":"2024-09-22T03:20:38.254223+0000","last_clean":"2024-09-22T03:20:38.254223+0000","last_became_active":"2024-09-22T03:20:15.157196+0000","last_became_peered":"2024-09-22T03:20:15.157196+0000","last_unstale":"2024-09-22T03:20:38.254223+0000","last_undegraded":"2024-09-22T03:20:38.254223+0000","last_fullsized":"2024-09-22T03:20:38.254223+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"106'791","reported_seq":824,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:39.174402+0000","last_change":"2024-09-22T03:20:15.256117+0000","last_active":"2024-09-22T03:20:39.174402+0000","last_peered":"2024-09-22T03:20:39.174402+0000","last_clean":"2024-09-22T03:20:39.174402+0000","last_became_active":"2024-09-22T03:20:15.158988+0000","last_became_peered":"2024-09-22T03:20:15.158988+0000","last_unstale":"2024-09-22T03:20:39.174402+0000","last_undegraded":"2024-09-22T03:20:39.174402+0000","last_fullsized":"2024-09-22T03:20:39.174402+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51904512,"num_objects":792,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":50688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"106'795","reported_seq":826,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:39.151882+0000","last_change":"2024-09-22T03:20:15.286545+0000","last_active":"2024-09-22T03:20:39.151882+0000","last_peered":"2024-09-22T03:20:39.151882+0000","last_clean":"2024-09-22T03:20:39.151882+0000","last_became_active":"2024-09-22T03:20:15.178048+0000","last_became_peered":"2024-09-22T03:20:15.178048+0000","last_unstale":"2024-09-22T03:20:39.151882+0000","last_undegraded":"2024-09-22T03:20:39.151882+0000","last_fullsized":"2024-09-22T03:20:39.151882+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":795,"log_dups_size":0,"ondisk_log_size":795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52166656,"num_objects":796,"num_object_clones":0,"num_object_copies":2388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":796,"num_write_kb":50944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"106'787","reported_seq":812,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:37.992125+0000","last_change":"2024-09-22T03:20:15.171913+0000","last_active":"2024-09-22T03:20:37.992125+0000","last_peered":"2024-09-22T03:20:37.992125+0000","last_clean":"2024-09-22T03:20:37.992125+0000","last_became_active":"2024-09-22T03:20:15.164720+0000","last_became_peered":"2024-09-22T03:20:15.164720+0000","last_unstale":"2024-09-22T03:20:37.992125+0000","last_undegraded":"2024-09-22T03:20:37.992125+0000","last_fullsized":"2024-09-22T03:20:37.992125+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51642368,"num_objects":788,"num_object_clones":0,"num_object_copies":2364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":788,"num_write_kb":50432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"106'762","reported_seq":797,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.964806+0000","last_change":"2024-09-22T03:20:15.292671+0000","last_active":"2024-09-22T03:20:38.964806+0000","last_peered":"2024-09-22T03:20:38.964806+0000","last_clean":"2024-09-22T03:20:38.964806+0000","last_became_active":"2024-09-22T03:20:15.171897+0000","last_became_peered":"2024-09-22T03:20:15.171897+0000","last_unstale":"2024-09-22T03:20:38.964806+0000","last_undegraded":"2024-09-22T03:20:38.964806+0000","last_fullsized":"2024-09-22T03:20:38.964806+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50003968,"num_objects":763,"num_object_clones":0,"num_object_copies":2289,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":763,"num_write_kb":48832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"106'742","reported_seq":768,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.255025+0000","last_change":"2024-09-22T03:20:15.174790+0000","last_active":"2024-09-22T03:20:38.255025+0000","last_peered":"2024-09-22T03:20:38.255025+0000","last_clean":"2024-09-22T03:20:38.255025+0000","last_became_active":"2024-09-22T03:20:15.164887+0000","last_became_peered":"2024-09-22T03:20:15.164887+0000","last_unstale":"2024-09-22T03:20:38.255025+0000","last_undegraded":"2024-09-22T03:20:38.255025+0000","last_fullsized":"2024-09-22T03:20:38.255025+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48693248,"num_objects":743,"num_object_clones":0,"num_object_copies":2229,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":47552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"106'808","reported_seq":833,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:40.345009+0000","last_change":"2024-09-22T03:19:42.473200+0000","last_active":"2024-09-22T03:20:40.345009+0000","last_peered":"2024-09-22T03:20:40.345009+0000","last_clean":"2024-09-22T03:20:40.345009+0000","last_became_active":"2024-09-22T03:19:42.468978+0000","last_became_peered":"2024-09-22T03:19:42.468978+0000","last_unstale":"2024-09-22T03:20:40.345009+0000","last_undegraded":"2024-09-22T03:20:40.345009+0000","last_fullsized":"2024-09-22T03:20:40.345009+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53018624,"num_objects":809,"num_object_clones":0,"num_object_copies":2427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":51776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"106'791","reported_seq":816,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.999934+0000","last_change":"2024-09-22T03:19:42.477912+0000","last_active":"2024-09-22T03:20:38.999934+0000","last_peered":"2024-09-22T03:20:38.999934+0000","last_clean":"2024-09-22T03:20:38.999934+0000","last_became_active":"2024-09-22T03:19:42.469937+0000","last_became_peered":"2024-09-22T03:19:42.469937+0000","last_unstale":"2024-09-22T03:20:38.999934+0000","last_undegraded":"2024-09-22T03:20:38.999934+0000","last_fullsized":"2024-09-22T03:20:38.999934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51904512,"num_objects":792,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":50688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"106'724","reported_seq":749,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.268591+0000","last_change":"2024-09-22T03:19:42.489336+0000","last_active":"2024-09-22T03:20:38.268591+0000","last_peered":"2024-09-22T03:20:38.268591+0000","last_clean":"2024-09-22T03:20:38.268591+0000","last_became_active":"2024-09-22T03:19:42.482335+0000","last_became_peered":"2024-09-22T03:19:42.482335+0000","last_unstale":"2024-09-22T03:20:38.268591+0000","last_undegraded":"2024-09-22T03:20:38.268591+0000","last_fullsized":"2024-09-22T03:20:38.268591+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47513600,"num_objects":725,"num_object_clones":0,"num_object_copies":2175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":46400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.085997+0000","last_change":"2024-09-22T03:20:20.093423+0000","last_active":"2024-09-22T03:20:20.085997+0000","last_peered":"2024-09-22T03:20:20.085997+0000","last_clean":"2024-09-22T03:20:20.085997+0000","last_became_active":"2024-09-22T03:20:15.171717+0000","last_became_peered":"2024-09-22T03:20:15.171717+0000","last_unstale":"2024-09-22T03:20:20.085997+0000","last_undegraded":"2024-09-22T03:20:20.085997+0000","last_fullsized":"2024-09-22T03:20:20.085997+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"106'783","reported_seq":814,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.286042+0000","last_change":"2024-09-22T03:20:15.325469+0000","last_active":"2024-09-22T03:20:38.286042+0000","last_peered":"2024-09-22T03:20:38.286042+0000","last_clean":"2024-09-22T03:20:38.286042+0000","last_became_active":"2024-09-22T03:20:15.188302+0000","last_became_peered":"2024-09-22T03:20:15.188302+0000","last_unstale":"2024-09-22T03:20:38.286042+0000","last_undegraded":"2024-09-22T03:20:38.286042+0000","last_fullsized":"2024-09-22T03:20:38.286042+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51380224,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":50176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":218,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.094938+0000","last_change":"2024-09-22T03:19:40.083887+0000","last_active":"2024-09-22T03:20:20.094938+0000","last_peered":"2024-09-22T03:20:20.094938+0000","last_clean":"2024-09-22T03:20:20.094938+0000","last_became_active":"2024-09-22T03:19:39.586255+0000","last_became_peered":"2024-09-22T03:19:39.586255+0000","last_unstale":"2024-09-22T03:20:20.094938+0000","last_undegraded":"2024-09-22T03:20:20.094938+0000","last_fullsized":"2024-09-22T03:20:20.094938+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"106'784","reported_seq":817,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.235126+0000","last_change":"2024-09-22T03:20:15.310737+0000","last_active":"2024-09-22T03:20:38.235126+0000","last_peered":"2024-09-22T03:20:38.235126+0000","last_clean":"2024-09-22T03:20:38.235126+0000","last_became_active":"2024-09-22T03:20:15.170038+0000","last_became_peered":"2024-09-22T03:20:15.170038+0000","last_unstale":"2024-09-22T03:20:38.235126+0000","last_undegraded":"2024-09-22T03:20:38.235126+0000","last_fullsized":"2024-09-22T03:20:38.235126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51445760,"num_objects":785,"num_object_clones":0,"num_object_copies":2355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":785,"num_write_kb":50240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":312,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.097504+0000","last_change":"2024-09-22T03:20:20.116543+0000","last_active":"2024-09-22T03:20:20.097504+0000","last_peered":"2024-09-22T03:20:20.097504+0000","last_clean":"2024-09-22T03:20:20.097504+0000","last_became_active":"2024-09-22T03:20:15.147909+0000","last_became_peered":"2024-09-22T03:20:15.147909+0000","last_unstale":"2024-09-22T03:20:20.097504+0000","last_undegraded":"2024-09-22T03:20:20.097504+0000","last_fullsized":"2024-09-22T03:20:20.097504+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"106'748","reported_seq":773,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:39.041284+0000","last_change":"2024-09-22T03:19:42.520982+0000","last_active":"2024-09-22T03:20:39.041284+0000","last_peered":"2024-09-22T03:20:39.041284+0000","last_clean":"2024-09-22T03:20:39.041284+0000","last_became_active":"2024-09-22T03:19:42.518086+0000","last_became_peered":"2024-09-22T03:19:42.518086+0000","last_unstale":"2024-09-22T03:20:39.041284+0000","last_undegraded":"2024-09-22T03:20:39.041284+0000","last_fullsized":"2024-09-22T03:20:39.041284+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49086464,"num_objects":749,"num_object_clones":0,"num_object_copies":2247,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":749,"num_write_kb":47936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":171,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.087763+0000","last_change":"2024-09-22T03:20:20.093627+0000","last_active":"2024-09-22T03:20:20.087763+0000","last_peered":"2024-09-22T03:20:20.087763+0000","last_clean":"2024-09-22T03:20:20.087763+0000","last_became_active":"2024-09-22T03:20:15.159573+0000","last_became_peered":"2024-09-22T03:20:15.159573+0000","last_unstale":"2024-09-22T03:20:20.087763+0000","last_undegraded":"2024-09-22T03:20:20.087763+0000","last_fullsized":"2024-09-22T03:20:20.087763+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":341,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.091705+0000","last_change":"2024-09-22T03:19:39.588674+0000","last_active":"2024-09-22T03:20:20.091705+0000","last_peered":"2024-09-22T03:20:20.091705+0000","last_clean":"2024-09-22T03:20:20.091705+0000","last_became_active":"2024-09-22T03:19:39.528532+0000","last_became_peered":"2024-09-22T03:19:39.528532+0000","last_unstale":"2024-09-22T03:20:20.091705+0000","last_undegraded":"2024-09-22T03:20:20.091705+0000","last_fullsized":"2024-09-22T03:20:20.091705+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"106'803","reported_seq":828,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:39.103768+0000","last_change":"2024-09-22T03:19:42.501027+0000","last_active":"2024-09-22T03:20:39.103768+0000","last_peered":"2024-09-22T03:20:39.103768+0000","last_clean":"2024-09-22T03:20:39.103768+0000","last_became_active":"2024-09-22T03:19:42.494283+0000","last_became_peered":"2024-09-22T03:19:42.494283+0000","last_unstale":"2024-09-22T03:20:39.103768+0000","last_undegraded":"2024-09-22T03:20:39.103768+0000","last_fullsized":"2024-09-22T03:20:39.103768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52690944,"num_objects":804,"num_object_clones":0,"num_object_copies":2412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":51456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.075953+0000","last_change":"2024-09-22T03:19:40.106505+0000","last_active":"2024-09-22T03:20:20.075953+0000","last_peered":"2024-09-22T03:20:20.075953+0000","last_clean":"2024-09-22T03:20:20.075953+0000","last_became_active":"2024-09-22T03:19:39.487648+0000","last_became_peered":"2024-09-22T03:19:39.487648+0000","last_unstale":"2024-09-22T03:20:20.075953+0000","last_undegraded":"2024-09-22T03:20:20.075953+0000","last_fullsized":"2024-09-22T03:20:20.075953+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"106'778","reported_seq":803,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.076633+0000","last_change":"2024-09-22T03:20:15.209720+0000","last_active":"2024-09-22T03:20:38.076633+0000","last_peered":"2024-09-22T03:20:38.076633+0000","last_clean":"2024-09-22T03:20:38.076633+0000","last_became_active":"2024-09-22T03:20:15.198961+0000","last_became_peered":"2024-09-22T03:20:15.198961+0000","last_unstale":"2024-09-22T03:20:38.076633+0000","last_undegraded":"2024-09-22T03:20:38.076633+0000","last_fullsized":"2024-09-22T03:20:38.076633+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51052544,"num_objects":779,"num_object_clones":0,"num_object_copies":2337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":779,"num_write_kb":49856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":298,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.103917+0000","last_change":"2024-09-22T03:20:20.111817+0000","last_active":"2024-09-22T03:20:20.103917+0000","last_peered":"2024-09-22T03:20:20.103917+0000","last_clean":"2024-09-22T03:20:20.103917+0000","last_became_active":"2024-09-22T03:20:15.168250+0000","last_became_peered":"2024-09-22T03:20:15.168250+0000","last_unstale":"2024-09-22T03:20:20.103917+0000","last_undegraded":"2024-09-22T03:20:20.103917+0000","last_fullsized":"2024-09-22T03:20:20.103917+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"106'689","reported_seq":715,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:39.164267+0000","last_change":"2024-09-22T03:20:15.149437+0000","last_active":"2024-09-22T03:20:39.164267+0000","last_peered":"2024-09-22T03:20:39.164267+0000","last_clean":"2024-09-22T03:20:39.164267+0000","last_became_active":"2024-09-22T03:20:15.144457+0000","last_became_peered":"2024-09-22T03:20:15.144457+0000","last_unstale":"2024-09-22T03:20:39.164267+0000","last_undegraded":"2024-09-22T03:20:39.164267+0000","last_fullsized":"2024-09-22T03:20:39.164267+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45219840,"num_objects":690,"num_object_clones":0,"num_object_copies":2070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":690,"num_write_kb":44160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":224,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.076259+0000","last_change":"2024-09-22T03:20:20.082089+0000","last_active":"2024-09-22T03:20:20.076259+0000","last_peered":"2024-09-22T03:20:20.076259+0000","last_clean":"2024-09-22T03:20:20.076259+0000","last_became_active":"2024-09-22T03:20:15.167293+0000","last_became_peered":"2024-09-22T03:20:15.167293+0000","last_unstale":"2024-09-22T03:20:20.076259+0000","last_undegraded":"2024-09-22T03:20:20.076259+0000","last_fullsized":"2024-09-22T03:20:20.076259+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"106'754","reported_seq":785,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:38.254065+0000","last_change":"2024-09-22T03:20:15.266705+0000","last_active":"2024-09-22T03:20:38.254065+0000","last_peered":"2024-09-22T03:20:38.254065+0000","last_clean":"2024-09-22T03:20:38.254065+0000","last_became_active":"2024-09-22T03:20:15.171141+0000","last_became_peered":"2024-09-22T03:20:15.171141+0000","last_unstale":"2024-09-22T03:20:38.254065+0000","last_undegraded":"2024-09-22T03:20:38.254065+0000","last_fullsized":"2024-09-22T03:20:38.254065+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49479680,"num_objects":755,"num_object_clones":0,"num_object_copies":2265,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":755,"num_write_kb":48320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":202,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-22T03:20:20.074349+0000","last_change":"2024-09-22T03:20:20.080550+0000","last_active":"2024-09-22T03:20:20.074349+0000","last_peered":"2024-09-22T03:20:20.074349+0000","last_clean":"2024-09-22T03:20:20.074349+0000","last_became_active":"2024-09-22T03:20:15.150206+0000","last_became_peered":"2024-09-22T03:20:15.150206+0000","last_unstale":"2024-09-22T03:20:20.074349+0000","last_undegraded":"2024-09-22T03:20:20.074349+0000","last_fullsized":"2024-09-22T03:20:20.074349+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":805437440,"num_objects":12290,"num_object_clones":0,"num_object_copies":36870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12290,"num_write_kb":786560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151556096,"data_stored":2424897536,"data_compressed":115143779,"data_compressed_allocated":151556096,"data_compressed_original":2424897536,"omap_allocated":0,"internal_metadata":0},"log_size":12274,"ondisk_log_size":12274,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676598808,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78652,"kb_used_data":33960,"kb_used_omap":0,"kb_used_meta":44672,"kb_avail":94293188,"statfs":{"total":96636764160,"available":96556224512,"internally_reserved":0,"allocated":34775040,"data_stored":542600720,"data_compressed":25747922,"data_compressed_allocated":33890304,"data_compressed_original":542244864,"omap_allocated":0,"internal_metadata":45744128},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673534,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":146108,"kb_used_data":42924,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94225732,"statfs":{"total":96636764160,"available":96487149568,"internally_reserved":0,"allocated":43954176,"data_stored":689074224,"data_compressed":32704987,"data_compressed_allocated":43048960,"data_compressed_original":688701440,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":95,"seq":408021893199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75896,"kb_used_data":32552,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94295944,"statfs":{"total":96636764160,"available":96559046656,"internally_reserved":0,"allocated":33333248,"data_stored":519139376,"data_compressed":24635857,"data_compressed_allocated":32428032,"data_compressed_original":518766592,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78356,"kb_used_data":40196,"kb_used_omap":0,"kb_used_meta":38144,"kb_avail":94293484,"statfs":{"total":96636764160,"available":96556527616,"internally_reserved":0,"allocated":41160704,"data_stored":644444189,"data_compressed":30585817,"data_compressed_allocated":40259584,"data_compressed_original":644071424,"omap_allocated":0,"internal_metadata":39059456},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41132032,"data_stored":658112512,"data_compressed":31249821,"data_compressed_allocated":41132032,"data_compressed_original":658112512,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32681984,"data_stored":522911744,"data_compressed":24829949,"data_compressed_allocated":32681984,"data_compressed_original":522911744,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43810816,"data_stored":700973056,"data_compressed":33284967,"data_compressed_allocated":43810816,"data_compressed_original":700973056,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33931264,"data_stored":542900224,"data_compressed":25779042,"data_compressed_allocated":33931264,"data_compressed_original":542900224,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:20:41.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:41.651+0000 7f20bef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:41.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:41.660+0000 7f20bef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:42.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 59 16 13593 13577 14.381 22.875 0.0263912 0.0695039 2024-09-22T03:20:43.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:20:43.490923+0000 min lat: 0.00842921 max lat: 23.3056 avg lat: 0.0688517 2024-09-22T03:20:43.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:20:43.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 60 16 13954 13938 14.5173 22.5625 0.0200817 0.0688517 2024-09-22T03:20:43.945 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:44.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:44.088+0000 7fb21846c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:44.097+0000 7fb21846c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:44.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 61 16 14296 14280 14.6297 21.375 0.0386046 0.0683253 2024-09-22T03:20:45.491 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 62 16 14647 14631 14.7475 21.9375 0.0603159 0.0677769 2024-09-22T03:20:46.316 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:46.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:46.455+0000 7f41a6a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:46.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:46.462+0000 7f41a6a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:46.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 63 16 15002 14986 14.8656 22.1875 0.0284113 0.0672326 2024-09-22T03:20:46.536 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:20:46.536 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:46.536 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:20:46.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:20:46.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:46.685+0000 7f2438b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:46.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:46.696+0000 7f2438b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:46.734 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:46.734 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:46.734 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:46.734 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:46.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:46.737 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:46.737 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:20:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:46.884+0000 7f5612d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:46.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:46.900+0000 7f5612d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:46.936 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:46.937 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:46.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:46.941 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:46.941 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:20:47.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:47.089+0000 7f9d15b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:47.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:47.101+0000 7f9d15b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:47.149 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:47.150 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:47.157 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:47.157 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:47.157 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:20:47.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:47.310+0000 7fc0e0b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:47.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:47.321+0000 7fc0e0b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:47.366 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:20:47.366 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:20:47.366 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:20:47.366 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:20:47.367 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:20:47.370 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:20:47.370 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:20:47.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 64 16 15307 15291 14.9311 19.0625 0.0587538 0.0669366 2024-09-22T03:20:48.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 65 16 15640 15624 15.0216 20.8125 0.0673373 0.0665455 2024-09-22T03:20:48.782 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:48.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:48.910+0000 7f7ef9957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:48.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:48.917+0000 7f7ef9957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:49.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 66 16 15977 15961 15.1131 21.0625 0.0663783 0.066145 2024-09-22T03:20:50.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 67 16 16317 16301 15.2047 21.25 0.0630881 0.0657406 2024-09-22T03:20:51.171 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:20:51.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:51.314+0000 7f34956fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:51.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:51.319+0000 7f34956fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:51.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 68 16 16641 16625 15.2788 20.25 0.0348173 0.0654211 2024-09-22T03:20:52.371 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 0, 3] 2024-09-22T03:20:52.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:20:52.371 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [2, 1, 0, 3] 2024-09-22T03:20:52.372 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:20:52.372 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:20:52.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 69 16 16936 16920 15.3246 18.4375 0.0590831 0.0651163 2024-09-22T03:20:53.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 70 16 16936 16920 15.1056 0 - 0.0651163 2024-09-22T03:20:53.614 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:20:53.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:53.719+0000 7fbc2570e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:53.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:20:53.724+0000 7fbc2570e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:20:54.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 71 16 16936 16920 14.8929 0 - 0.0651163 2024-09-22T03:20:55.492 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 72 16 16936 16920 14.686 0 - 0.0651163 2024-09-22T03:20:56.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 73 16 16936 16920 14.4849 0 - 0.0651163 2024-09-22T03:20:57.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:20:57.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 74 16 16936 16920 14.2891 0 - 0.0651163 2024-09-22T03:20:58.473 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:20:58.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 75 16 16936 16920 14.0986 0 - 0.0651163 2024-09-22T03:20:59.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 76 16 16936 16920 13.9131 0 - 0.0651163 2024-09-22T03:21:00.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 77 16 16936 16920 13.7324 0 - 0.0651163 2024-09-22T03:21:01.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 78 16 16936 16920 13.5563 0 - 0.0651163 2024-09-22T03:21:02.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 79 16 16936 16920 13.3847 0 - 0.0651163 2024-09-22T03:21:02.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:21:03.474 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [1, 0, 3] 2024-09-22T03:21:03.474 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:21:03.474 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-22T03:21:03.474 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:21:03.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:21:03.477 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:21:03.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:21:03.493 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:21:03.493091+0000 min lat: 0.00842921 max lat: 23.3056 avg lat: 0.0651163 2024-09-22T03:21:03.494 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:21:03.494 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 80 16 16936 16920 13.2174 0 - 0.0651163 2024-09-22T03:21:03.587 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:21:03.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:21:03.591 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:21:04.494 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 81 16 16936 16920 13.0542 0 - 0.0651163 2024-09-22T03:21:05.494 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 82 16 16936 16920 12.895 0 - 0.0651163 2024-09-22T03:21:06.494 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 83 16 16936 16920 12.7396 0 - 0.0651163 2024-09-22T03:21:07.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 84 16 16936 16920 12.588 0 - 0.0651163 2024-09-22T03:21:08.494 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 85 16 16936 16920 12.4399 0 - 0.0651163 2024-09-22T03:21:08.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:21:08.700 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:21:08.703 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:21:08.704 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:21:09.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 86 16 16936 16920 12.2952 0 - 0.0651163 2024-09-22T03:21:10.494 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 87 16 16936 16920 12.1539 0 - 0.0651163 2024-09-22T03:21:11.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 88 16 16936 16920 12.0158 0 - 0.0651163 2024-09-22T03:21:12.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 89 16 16936 16920 11.8808 0 - 0.0651163 2024-09-22T03:21:13.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 90 16 16936 16920 11.7488 0 - 0.0651163 2024-09-22T03:21:13.704 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:21:13.803 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:21:13.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:21:13.806 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:21:14.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 91 16 16936 16920 11.6196 0 - 0.0651163 2024-09-22T03:21:15.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 92 16 16936 16920 11.4933 0 - 0.0651163 2024-09-22T03:21:16.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 93 16 16936 16920 11.3697 0 - 0.0651163 2024-09-22T03:21:17.495 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 94 16 16936 16920 11.2488 0 - 0.0651163 2024-09-22T03:21:18.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 95 16 16936 16920 11.1304 0 - 0.0651163 2024-09-22T03:21:18.807 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:21:18.936 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:21:18.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:21:18.939 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:21:19.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 96 16 16936 16920 11.0144 0 - 0.0651163 2024-09-22T03:21:20.077 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:21:20.147 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:21:20.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:21:20.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 97 16 16936 16920 10.9009 0 - 0.0651163 2024-09-22T03:21:21.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 98 16 16936 16920 10.7896 0 - 0.0651163 2024-09-22T03:21:22.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:21:22.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:22.278+0000 7faa4c6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:22.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:22.283+0000 7faa4c6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:22.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 99 6 16936 16930 10.687 0.0208333 29.914 0.0827392 2024-09-22T03:21:23.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:21:23.495888+0000 min lat: 0.00842921 max lat: 29.9283 avg lat: 0.0827392 2024-09-22T03:21:23.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:21:23.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 100 6 16936 16930 10.5801 0 - 0.0827392 2024-09-22T03:21:23.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:21:24.052 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:21:24.052 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [] 2024-09-22T03:21:24.052 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:21:24.356 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:21:24.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:24.469+0000 7f8b33d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:24.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:24.474+0000 7f8b33d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:24.496 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 101 6 16936 16930 10.4753 0 - 0.0827392 2024-09-22T03:21:25.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 102 6 16936 16930 10.3726 0 - 0.0827392 2024-09-22T03:21:26.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 103 6 16936 16930 10.2719 0 - 0.0827392 2024-09-22T03:21:26.544 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:21:26.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:26.657+0000 7fdb0eed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:26.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:26.662+0000 7fdb0eed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:27.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 104 6 16936 16930 10.1732 0 - 0.0827392 2024-09-22T03:21:28.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 105 6 16936 16930 10.0763 0 - 0.0827392 2024-09-22T03:21:28.734 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:21:28.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:28.847+0000 7f69ccb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:28.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:28.851+0000 7f69ccb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:29.062 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [1, 0, 3, 2] 2024-09-22T03:21:29.062 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:21:29.063 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [1, 0, 3, 2] 2024-09-22T03:21:29.063 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:21:29.063 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:21:29.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 106 6 16936 16930 9.98121 0 - 0.0827392 2024-09-22T03:21:30.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 107 6 16936 16930 9.88792 0 - 0.0827392 2024-09-22T03:21:30.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:21:30.924 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:21:31.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:31.037+0000 7f2234357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:31.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:31.042+0000 7f2234357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:31.497 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 108 6 16936 16930 9.79636 0 - 0.0827392 2024-09-22T03:21:32.498 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 109 6 16936 16930 9.70648 0 - 0.0827392 2024-09-22T03:21:33.498 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 110 6 16936 16930 9.61824 0 - 0.0827392 2024-09-22T03:21:34.498 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 111 6 16936 16930 9.53159 0 - 0.0827392 2024-09-22T03:21:35.164 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:21:35.498 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 112 6 16936 16930 9.44648 0 - 0.0827392 2024-09-22T03:21:36.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:21:36.498 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 113 6 16936 16930 9.36288 0 - 0.0827392 2024-09-22T03:21:37.498 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 114 6 16936 16930 9.28074 0 - 0.0827392 2024-09-22T03:21:38.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 115 6 16936 16930 9.20004 0 - 0.0827392 2024-09-22T03:21:39.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 116 6 16936 16930 9.12072 0 - 0.0827392 2024-09-22T03:21:40.165 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [1, 0, 3] 2024-09-22T03:21:40.165 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:21:40.165 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [1, 0, 3] 2024-09-22T03:21:40.165 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:21:40.166 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:21:40.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 117 6 16936 16930 9.04276 0 - 0.0827392 2024-09-22T03:21:41.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 118 6 16936 16930 8.96613 0 - 0.0827392 2024-09-22T03:21:41.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:21:42.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 119 6 16936 16930 8.89078 0 - 0.0827392 2024-09-22T03:21:43.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:21:43.498842+0000 min lat: 0.00842921 max lat: 29.9283 avg lat: 0.0827392 2024-09-22T03:21:43.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:21:43.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 120 6 16936 16930 8.81669 0 - 0.0827392 2024-09-22T03:21:44.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 121 6 16936 16930 8.74382 0 - 0.0827392 2024-09-22T03:21:45.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 122 6 16936 16930 8.67215 0 - 0.0827392 2024-09-22T03:21:46.266 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:21:46.499 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 123 6 16936 16930 8.60165 0 - 0.0827392 2024-09-22T03:21:47.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:21:47.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 124 6 16936 16930 8.53228 0 - 0.0827392 2024-09-22T03:21:48.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 125 6 16936 16930 8.46402 0 - 0.0827392 2024-09-22T03:21:49.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 126 6 16936 16930 8.39684 0 - 0.0827392 2024-09-22T03:21:50.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 127 6 16936 16930 8.33072 0 - 0.0827392 2024-09-22T03:21:51.266 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2, 1] live_osds: [0, 3] 2024-09-22T03:21:51.267 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:21:51.267 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:21:51.267 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:21:51.267 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:21:51.270 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:21:51.270 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:21:51.380 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:21:51.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:21:51.385 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:21:51.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 128 6 16936 16930 8.26564 0 - 0.0827392 2024-09-22T03:21:52.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 129 6 16936 16930 8.20157 0 - 0.0827392 2024-09-22T03:21:53.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 130 6 16936 16930 8.13848 0 - 0.0827392 2024-09-22T03:21:54.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 131 6 16936 16930 8.07635 0 - 0.0827392 2024-09-22T03:21:55.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 132 6 16936 16930 8.01517 0 - 0.0827392 2024-09-22T03:21:56.386 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:21:56.483 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:21:56.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:21:56.486 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:21:56.500 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 133 6 16936 16930 7.95491 0 - 0.0827392 2024-09-22T03:21:57.087 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:21:57.163 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:21:57.163 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:21:57.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 134 6 16936 16930 7.89554 0 - 0.0827392 2024-09-22T03:21:58.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 135 3 16936 16933 7.83845 0.00520833 65.7838 0.0943783 2024-09-22T03:21:59.164 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:21:59.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:59.266+0000 7fb6d0797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:59.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:21:59.270+0000 7fb6d0797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:21:59.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 136 3 16936 16933 7.78081 0 - 0.0943783 2024-09-22T03:22:00.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 137 3 16936 16933 7.72402 0 - 0.0943783 2024-09-22T03:22:01.486 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:22:01.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 138 3 16936 16933 7.66805 0 - 0.0943783 2024-09-22T03:22:01.582 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:22:01.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:01.585 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:22:02.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 139 3 16936 16933 7.61288 0 - 0.0943783 2024-09-22T03:22:03.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:22:03.501095+0000 min lat: 0.00842921 max lat: 65.7838 avg lat: 0.0943783 2024-09-22T03:22:03.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:22:03.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 140 3 16936 16933 7.5585 0 - 0.0943783 2024-09-22T03:22:04.275 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:22:04.275 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:04.294 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:04.294 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:04.501 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 141 3 16936 16933 7.5049 0 - 0.0943783 2024-09-22T03:22:05.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 142 3 16936 16933 7.45204 0 - 0.0943783 2024-09-22T03:22:06.295 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:06.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:06.396+0000 7fc57a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:06.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:06.400+0000 7fc57a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:06.448 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:06.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total time run: 142.725 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total writes made: 16936 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Write size: 65536 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object size: 65536 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Bandwidth (MB/sec): 7.41635 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Bandwidth: 10.81 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max bandwidth (MB/sec): 25.9375 2024-09-22T03:22:06.502 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min bandwidth (MB/sec): 0 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average IOPS: 118 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev IOPS: 172.962 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max IOPS: 415 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min IOPS: 0 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average Latency(s): 0.107439 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Latency(s): 1.66194 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max latency(s): 73.8403 2024-09-22T03:22:06.503 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min latency(s): 0.00842921 2024-09-22T03:22:06.517 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:22:06.587 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 143, 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.691 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.526756+0000" 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.526771+0000" 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.526930+0000" 2024-09-22T03:22:06.692 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:22:06.693 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.527020+0000" 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.527280+0000" 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.694 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.560140+0000" 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:22:06.695 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.527617+0000" 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.696 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.527885+0000" 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.697 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.528195+0000" 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.698 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.699 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.531204+0000" 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:22:06.700 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.532810+0000" 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.701 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.559669+0000" 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.702 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 165, 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.584320+0000" 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.703 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.584325+0000" 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.584404+0000" 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.704 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.584483+0000" 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.705 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.584800+0000" 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.706 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.602659+0000" 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.585112+0000" 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:22:06.707 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.585282+0000" 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.708 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.585446+0000" 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.709 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.710 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.585680+0000" 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.711 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.592027+0000" 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.712 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:22:06.602362+0000" 2024-09-22T03:22:06.713 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:22:06.714 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:22:08.449 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:08.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:08.569+0000 7f5198397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:08.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:08.575+0000 7f5198397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:08.708 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:08.728 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:08.729 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:10.730 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:10.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:10.852+0000 7f57ab42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:10.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:10.858+0000 7f57ab42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:10.980 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:11.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:11.006 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:11.716 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [0, 3, 1] 2024-09-22T03:22:11.716 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:22:11.716 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-22T03:22:11.716 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:22:11.716 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:22:11.719 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:22:11.719 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:22:11.836 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:22:11.839 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:11.840 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:22:13.006 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:13.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:13.141+0000 7fbc9b2d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:13.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:13.146+0000 7fbc9b2d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:13.255 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:13.276 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:13.276 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:15.277 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:15.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:15.408+0000 7f6ea2d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:15.416 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:15.414+0000 7f6ea2d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:15.565 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:15.606 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:15.606 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:16.841 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:22:16.949 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:22:16.952 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:16.952 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:22:17.608 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:17.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:17.734+0000 7f7b22afa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:17.740+0000 7f7b22afa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:17.851 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:17.985 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:17.986 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:19.986 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:20.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:20.115+0000 7f827e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:20.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:20.122+0000 7f827e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:20.250 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:22:20.289 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:22:20.289 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:22:21.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:22:22.065 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:22:22.065 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [] 2024-09-22T03:22:22.066 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:22:22.291 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:22.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:22.428+0000 7f230c117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:22.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:22.436+0000 7f230c117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:24.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:24.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:24.718+0000 7f1c9384f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:24.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:24.723+0000 7f1c9384f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:26.939 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:27.079 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:22:27.079 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:22:27.079 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:22:27.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:27.080+0000 7f6658c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:27.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:27.090+0000 7f6658c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:27.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:27.207+0000 7ffb6862c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:27.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:27.218+0000 7ffb6862c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:27.452 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:22:27.452 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:22:27.469 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-22T03:22:27.369287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":815989323,"num_objects":12447,"num_object_clones":0,"num_object_copies":37341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":834,"num_objects_misplaced":831,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":21570,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":16449536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21530,"ondisk_log_size":21530,"up":72,"acting":61,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":335800,"kb_used_data":168760,"kb_used_omap":0,"kb_used_meta":166976,"kb_avail":377151560,"statfs":{"total":386547056640,"available":386203197440,"internally_reserved":0,"allocated":172810240,"data_stored":2702909711,"data_compressed":128278433,"data_compressed_allocated":168845312,"data_compressed_original":2701279232,"omap_allocated":0,"internal_metadata":170983424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-62128128,"num_objects":-948,"num_object_clones":0,"num_object_copies":-2844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-10977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":948,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.939066"},"pg_stats":[{"pgid":"5.f","version":"118'1289","reported_seq":1345,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.465421+0000","last_change":"2024-09-22T03:22:23.465421+0000","last_active":"2024-09-22T03:22:23.465421+0000","last_peered":"2024-09-22T03:22:22.406183+0000","last_clean":"2024-09-22T03:20:52.279388+0000","last_became_active":"2024-09-22T03:21:58.129288+0000","last_became_peered":"2024-09-22T03:21:58.129288+0000","last_unstale":"2024-09-22T03:22:23.465421+0000","last_undegraded":"2024-09-22T03:22:23.465421+0000","last_fullsized":"2024-09-22T03:22:23.465421+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1289,"log_dups_size":0,"ondisk_log_size":1289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49610752,"num_objects":757,"num_object_clones":0,"num_object_copies":2271,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1289,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.e","version":"118'1347","reported_seq":1414,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.443660+0000","last_change":"2024-09-22T03:22:23.443660+0000","last_active":"2024-09-22T03:22:22.479940+0000","last_peered":"2024-09-22T03:22:22.440548+0000","last_clean":"2024-09-22T03:21:25.082463+0000","last_became_active":"2024-09-22T03:21:58.125578+0000","last_became_peered":"2024-09-22T03:21:58.125578+0000","last_unstale":"2024-09-22T03:22:23.443660+0000","last_undegraded":"2024-09-22T03:22:23.443660+0000","last_fullsized":"2024-09-22T03:22:23.443660+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1347,"log_dups_size":0,"ondisk_log_size":1347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49610752,"num_objects":757,"num_object_clones":0,"num_object_copies":2271,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1347,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"118'1355","reported_seq":1428,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.453867+0000","last_change":"2024-09-22T03:22:23.453867+0000","last_active":"2024-09-22T03:22:22.489105+0000","last_peered":"2024-09-22T03:22:22.351574+0000","last_clean":"2024-09-22T03:21:25.086946+0000","last_became_active":"2024-09-22T03:21:58.131591+0000","last_became_peered":"2024-09-22T03:21:58.131591+0000","last_unstale":"2024-09-22T03:22:23.453867+0000","last_undegraded":"2024-09-22T03:22:23.453867+0000","last_fullsized":"2024-09-22T03:22:23.453867+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1355,"log_dups_size":0,"ondisk_log_size":1355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50462744,"num_objects":771,"num_object_clones":0,"num_object_copies":2313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1355,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"118'1370","reported_seq":1422,"reported_epoch":119,"state":"peering","last_fresh":"2024-09-22T03:22:22.404888+0000","last_change":"2024-09-22T03:22:22.440052+0000","last_active":"2024-09-22T03:22:22.440052+0000","last_peered":"2024-09-22T03:22:22.404888+0000","last_clean":"2024-09-22T03:21:25.083659+0000","last_became_active":"2024-09-22T03:22:06.127319+0000","last_became_peered":"2024-09-22T03:22:06.127319+0000","last_unstale":"2024-09-22T03:22:22.404888+0000","last_undegraded":"2024-09-22T03:22:06.095176+0000","last_fullsized":"2024-09-22T03:22:06.094326+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1370,"log_dups_size":0,"ondisk_log_size":1370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50855936,"num_objects":776,"num_object_clones":0,"num_object_copies":2328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1370,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"118'1350","reported_seq":1427,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T03:22:23.450624+0000","last_change":"2024-09-22T03:22:06.176976+0000","last_active":"2024-09-22T03:22:23.450624+0000","last_peered":"2024-09-22T03:22:23.450624+0000","last_clean":"2024-09-22T03:22:23.450624+0000","last_became_active":"2024-09-22T03:22:06.172885+0000","last_became_peered":"2024-09-22T03:22:06.172885+0000","last_unstale":"2024-09-22T03:22:23.450624+0000","last_undegraded":"2024-09-22T03:22:23.450624+0000","last_fullsized":"2024-09-22T03:22:23.450624+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1350,"log_dups_size":0,"ondisk_log_size":1350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49414144,"num_objects":754,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1350,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"118'1334","reported_seq":1381,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.453549+0000","last_change":"2024-09-22T03:22:23.453549+0000","last_active":"2024-09-22T03:22:23.453549+0000","last_peered":"2024-09-22T03:22:22.338095+0000","last_clean":"2024-09-22T03:20:52.363428+0000","last_became_active":"2024-09-22T03:21:58.134631+0000","last_became_peered":"2024-09-22T03:21:58.134631+0000","last_unstale":"2024-09-22T03:22:23.453549+0000","last_undegraded":"2024-09-22T03:22:23.453549+0000","last_fullsized":"2024-09-22T03:22:23.453549+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51511296,"num_objects":786,"num_object_clones":0,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1334,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"118'1364","reported_seq":1410,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.454409+0000","last_change":"2024-09-22T03:22:23.454409+0000","last_active":"2024-09-22T03:22:22.459642+0000","last_peered":"2024-09-22T03:22:22.426699+0000","last_clean":"2024-09-22T03:21:22.194491+0000","last_became_active":"2024-09-22T03:22:06.169327+0000","last_became_peered":"2024-09-22T03:22:06.169327+0000","last_unstale":"2024-09-22T03:22:23.454409+0000","last_undegraded":"2024-09-22T03:22:23.454409+0000","last_fullsized":"2024-09-22T03:22:23.454409+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1364,"log_dups_size":0,"ondisk_log_size":1364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50855936,"num_objects":776,"num_object_clones":0,"num_object_copies":2328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1364,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"118'1291","reported_seq":1802,"reported_epoch":118,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:22:19.205052+0000","last_change":"2024-09-22T03:22:07.374671+0000","last_active":"2024-09-22T03:22:19.205052+0000","last_peered":"2024-09-22T03:22:19.205052+0000","last_clean":"2024-09-22T03:21:25.083073+0000","last_became_active":"2024-09-22T03:22:07.323664+0000","last_became_peered":"2024-09-22T03:22:07.323664+0000","last_unstale":"2024-09-22T03:22:19.205052+0000","last_undegraded":"2024-09-22T03:22:07.150744+0000","last_fullsized":"2024-09-22T03:22:07.118326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1291,"log_dups_size":0,"ondisk_log_size":1291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54591488,"num_objects":833,"num_object_clones":0,"num_object_copies":2499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":833,"num_objects_misplaced":831,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":15138816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"118'1292","reported_seq":1333,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.456418+0000","last_change":"2024-09-22T03:22:23.456418+0000","last_active":"2024-09-22T03:22:23.456418+0000","last_peered":"2024-09-22T03:22:22.435066+0000","last_clean":"2024-09-22T03:20:52.350718+0000","last_became_active":"2024-09-22T03:22:06.173275+0000","last_became_peered":"2024-09-22T03:22:06.173275+0000","last_unstale":"2024-09-22T03:22:23.456418+0000","last_undegraded":"2024-09-22T03:22:23.456418+0000","last_fullsized":"2024-09-22T03:22:23.456418+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1292,"log_dups_size":0,"ondisk_log_size":1292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49020928,"num_objects":748,"num_object_clones":0,"num_object_copies":2244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1292,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":125,"reported_epoch":119,"state":"peering","last_fresh":"2024-09-22T03:22:07.120984+0000","last_change":"2024-09-22T03:22:22.457807+0000","last_active":"2024-09-22T03:22:22.457807+0000","last_peered":"2024-09-22T03:22:07.120984+0000","last_clean":"2024-09-22T03:21:25.089690+0000","last_became_active":"2024-09-22T03:21:58.121434+0000","last_became_peered":"2024-09-22T03:21:58.121434+0000","last_unstale":"2024-09-22T03:22:07.120984+0000","last_undegraded":"2024-09-22T03:22:07.120984+0000","last_fullsized":"2024-09-22T03:21:58.102266+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"118'1387","reported_seq":1434,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.452023+0000","last_change":"2024-09-22T03:22:23.452023+0000","last_active":"2024-09-22T03:22:23.452023+0000","last_peered":"2024-09-22T03:22:22.292923+0000","last_clean":"2024-09-22T03:20:52.337102+0000","last_became_active":"2024-09-22T03:22:06.134269+0000","last_became_peered":"2024-09-22T03:22:06.134269+0000","last_unstale":"2024-09-22T03:22:23.452023+0000","last_undegraded":"2024-09-22T03:22:23.452023+0000","last_fullsized":"2024-09-22T03:22:23.452023+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1387,"log_dups_size":0,"ondisk_log_size":1387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50397184,"num_objects":769,"num_object_clones":0,"num_object_copies":2307,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1387,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":255,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T03:22:23.558632+0000","last_change":"2024-09-22T03:22:23.559129+0000","last_active":"2024-09-22T03:22:23.558632+0000","last_peered":"2024-09-22T03:22:23.558632+0000","last_clean":"2024-09-22T03:21:25.089165+0000","last_became_active":"2024-09-22T03:22:23.553334+0000","last_became_peered":"2024-09-22T03:22:23.553334+0000","last_unstale":"2024-09-22T03:22:23.558632+0000","last_undegraded":"2024-09-22T03:22:23.558632+0000","last_fullsized":"2024-09-22T03:22:23.558632+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"118'1402","reported_seq":1456,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.449297+0000","last_change":"2024-09-22T03:22:23.449297+0000","last_active":"2024-09-22T03:22:23.449297+0000","last_peered":"2024-09-22T03:22:22.399600+0000","last_clean":"2024-09-22T03:20:52.366407+0000","last_became_active":"2024-09-22T03:21:58.124106+0000","last_became_peered":"2024-09-22T03:21:58.124106+0000","last_unstale":"2024-09-22T03:22:23.449297+0000","last_undegraded":"2024-09-22T03:22:23.449297+0000","last_fullsized":"2024-09-22T03:22:23.449297+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1402,"log_dups_size":0,"ondisk_log_size":1402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51642368,"num_objects":788,"num_object_clones":0,"num_object_copies":2364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1402,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":118,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:22:07.114114+0000","last_change":"2024-09-22T03:22:06.130395+0000","last_active":"2024-09-22T03:22:07.114114+0000","last_peered":"2024-09-22T03:22:07.114114+0000","last_clean":"2024-09-22T03:11:08.281767+0000","last_became_active":"2024-09-22T03:22:06.130395+0000","last_became_peered":"2024-09-22T03:22:06.130395+0000","last_unstale":"2024-09-22T03:22:07.114114+0000","last_undegraded":"2024-09-22T03:22:06.099584+0000","last_fullsized":"2024-09-22T03:22:06.098743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"118'1306","reported_seq":1369,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.460816+0000","last_change":"2024-09-22T03:22:23.460816+0000","last_active":"2024-09-22T03:22:22.473078+0000","last_peered":"2024-09-22T03:22:22.452003+0000","last_clean":"2024-09-22T03:21:25.087190+0000","last_became_active":"2024-09-22T03:22:06.180873+0000","last_became_peered":"2024-09-22T03:22:06.180873+0000","last_unstale":"2024-09-22T03:22:23.460816+0000","last_undegraded":"2024-09-22T03:22:23.460816+0000","last_fullsized":"2024-09-22T03:22:23.460816+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50200576,"num_objects":766,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1306,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":212,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:22:22.459763+0000","last_change":"2024-09-22T03:22:07.122258+0000","last_active":"2024-09-22T03:22:22.459763+0000","last_peered":"2024-09-22T03:22:22.459763+0000","last_clean":"2024-09-22T03:22:22.459763+0000","last_became_active":"2024-09-22T03:22:06.153374+0000","last_became_peered":"2024-09-22T03:22:06.153374+0000","last_unstale":"2024-09-22T03:22:22.459763+0000","last_undegraded":"2024-09-22T03:22:22.459763+0000","last_fullsized":"2024-09-22T03:22:22.459763+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":377,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T03:22:23.559941+0000","last_change":"2024-09-22T03:22:23.560509+0000","last_active":"2024-09-22T03:22:23.559941+0000","last_peered":"2024-09-22T03:22:23.559941+0000","last_clean":"2024-09-22T03:21:25.087682+0000","last_became_active":"2024-09-22T03:22:23.555962+0000","last_became_peered":"2024-09-22T03:22:23.555962+0000","last_unstale":"2024-09-22T03:22:23.559941+0000","last_undegraded":"2024-09-22T03:22:23.559941+0000","last_fullsized":"2024-09-22T03:22:23.559941+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"118'1411","reported_seq":1472,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.466472+0000","last_change":"2024-09-22T03:22:23.466472+0000","last_active":"2024-09-22T03:22:22.483949+0000","last_peered":"2024-09-22T03:22:22.336151+0000","last_clean":"2024-09-22T03:21:25.087309+0000","last_became_active":"2024-09-22T03:22:06.178428+0000","last_became_peered":"2024-09-22T03:22:06.178428+0000","last_unstale":"2024-09-22T03:22:23.466472+0000","last_undegraded":"2024-09-22T03:22:23.466472+0000","last_fullsized":"2024-09-22T03:22:23.466472+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52363264,"num_objects":799,"num_object_clones":0,"num_object_copies":2397,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T03:22:23.554992+0000","last_change":"2024-09-22T03:22:23.555774+0000","last_active":"2024-09-22T03:22:23.554992+0000","last_peered":"2024-09-22T03:22:23.554992+0000","last_clean":"2024-09-22T03:21:22.182896+0000","last_became_active":"2024-09-22T03:22:23.552128+0000","last_became_peered":"2024-09-22T03:22:23.552128+0000","last_unstale":"2024-09-22T03:22:23.554992+0000","last_undegraded":"2024-09-22T03:22:23.554992+0000","last_fullsized":"2024-09-22T03:22:23.554992+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"118'1393","reported_seq":1450,"reported_epoch":119,"state":"peering","last_fresh":"2024-09-22T03:22:22.362424+0000","last_change":"2024-09-22T03:22:22.459190+0000","last_active":"2024-09-22T03:22:22.459190+0000","last_peered":"2024-09-22T03:22:22.362424+0000","last_clean":"2024-09-22T03:21:25.087772+0000","last_became_active":"2024-09-22T03:21:58.124127+0000","last_became_peered":"2024-09-22T03:21:58.124127+0000","last_unstale":"2024-09-22T03:22:22.362424+0000","last_undegraded":"2024-09-22T03:21:58.104831+0000","last_fullsized":"2024-09-22T03:21:58.104047+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1393,"log_dups_size":0,"ondisk_log_size":1393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53018624,"num_objects":809,"num_object_clones":0,"num_object_copies":2427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1393,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":315,"reported_epoch":118,"state":"active+undersized","last_fresh":"2024-09-22T03:22:07.114519+0000","last_change":"2024-09-22T03:21:58.119851+0000","last_active":"2024-09-22T03:22:07.114519+0000","last_peered":"2024-09-22T03:22:07.114519+0000","last_clean":"2024-09-22T03:20:15.178249+0000","last_became_active":"2024-09-22T03:21:58.119851+0000","last_became_peered":"2024-09-22T03:21:58.119851+0000","last_unstale":"2024-09-22T03:22:07.114519+0000","last_undegraded":"2024-09-22T03:22:07.114519+0000","last_fullsized":"2024-09-22T03:21:58.096543+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"118'1216","reported_seq":1285,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T03:22:23.449949+0000","last_change":"2024-09-22T03:22:06.164385+0000","last_active":"2024-09-22T03:22:23.449949+0000","last_peered":"2024-09-22T03:22:23.449949+0000","last_clean":"2024-09-22T03:22:23.449949+0000","last_became_active":"2024-09-22T03:22:06.159146+0000","last_became_peered":"2024-09-22T03:22:06.159146+0000","last_unstale":"2024-09-22T03:22:23.449949+0000","last_undegraded":"2024-09-22T03:22:23.449949+0000","last_fullsized":"2024-09-22T03:22:23.449949+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48955392,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1217,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":252,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T03:22:23.554422+0000","last_change":"2024-09-22T03:22:23.555245+0000","last_active":"2024-09-22T03:22:23.554422+0000","last_peered":"2024-09-22T03:22:23.554422+0000","last_clean":"2024-09-22T03:21:22.127259+0000","last_became_active":"2024-09-22T03:22:23.551194+0000","last_became_peered":"2024-09-22T03:22:23.551194+0000","last_unstale":"2024-09-22T03:22:23.554422+0000","last_undegraded":"2024-09-22T03:22:23.554422+0000","last_fullsized":"2024-09-22T03:22:23.554422+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"118'1323","reported_seq":1370,"reported_epoch":120,"state":"remapped+peering","last_fresh":"2024-09-22T03:22:23.444287+0000","last_change":"2024-09-22T03:22:23.444287+0000","last_active":"2024-09-22T03:22:23.444287+0000","last_peered":"2024-09-22T03:22:22.396082+0000","last_clean":"2024-09-22T03:20:52.284271+0000","last_became_active":"2024-09-22T03:22:06.132738+0000","last_became_peered":"2024-09-22T03:22:06.132738+0000","last_unstale":"2024-09-22T03:22:23.444287+0000","last_undegraded":"2024-09-22T03:22:23.444287+0000","last_fullsized":"2024-09-22T03:22:23.444287+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52887552,"num_objects":807,"num_object_clones":0,"num_object_copies":2421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1323,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":217,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-22T03:22:23.453284+0000","last_change":"2024-09-22T03:22:07.120565+0000","last_active":"2024-09-22T03:22:23.453284+0000","last_peered":"2024-09-22T03:22:23.453284+0000","last_clean":"2024-09-22T03:22:23.453284+0000","last_became_active":"2024-09-22T03:22:06.159169+0000","last_became_peered":"2024-09-22T03:22:06.159169+0000","last_unstale":"2024-09-22T03:22:23.453284+0000","last_undegraded":"2024-09-22T03:22:23.453284+0000","last_fullsized":"2024-09-22T03:22:23.453284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":815398936,"num_objects":12443,"num_object_clones":0,"num_object_copies":37329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":833,"num_objects_misplaced":831,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":21431,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":16449536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168542208,"data_stored":2696544304,"data_compressed":128045365,"data_compressed_allocated":168534016,"data_compressed_original":2696544256,"omap_allocated":0,"internal_metadata":0},"log_size":21430,"ondisk_log_size":21430,"up":47,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":22,"acting":22,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676598912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75764,"kb_used_data":35236,"kb_used_omap":0,"kb_used_meta":40512,"kb_avail":94296076,"statfs":{"total":96636764160,"available":96559181824,"internally_reserved":0,"allocated":36081664,"data_stored":561910248,"data_compressed":26663615,"data_compressed_allocated":35094528,"data_compressed_original":561512448,"omap_allocated":0,"internal_metadata":41484288},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100764,"kb_used_data":60556,"kb_used_omap":0,"kb_used_meta":40192,"kb_avail":94271076,"statfs":{"total":96636764160,"available":96533581824,"internally_reserved":0,"allocated":62009344,"data_stored":977446868,"data_compressed":46397787,"data_compressed_allocated":61071360,"data_compressed_original":977059840,"omap_allocated":0,"internal_metadata":41156608},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":116,"seq":498216206355,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66112,"kb_used_data":31856,"kb_used_omap":0,"kb_used_meta":34240,"kb_avail":94305728,"statfs":{"total":96636764160,"available":96569065472,"internally_reserved":0,"allocated":32620544,"data_stored":505951143,"data_compressed":24007932,"data_compressed_allocated":31600640,"data_compressed_original":505528320,"omap_allocated":0,"internal_metadata":35061760},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93160,"kb_used_data":41112,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94278680,"statfs":{"total":96636764160,"available":96541368320,"internally_reserved":0,"allocated":42098688,"data_stored":657601452,"data_compressed":31209099,"data_compressed_allocated":41078784,"data_compressed_original":657178624,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41041920,"data_stored":656605208,"data_compressed":31179127,"data_compressed_allocated":41037824,"data_compressed_original":656605184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31559680,"data_stored":504954880,"data_compressed":23977960,"data_compressed_allocated":31559680,"data_compressed_original":504954880,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61030400,"data_stored":976486400,"data_compressed":46367815,"data_compressed_allocated":61030400,"data_compressed_original":976486400,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34910208,"data_stored":558497816,"data_compressed":26520463,"data_compressed_allocated":34906112,"data_compressed_original":558497792,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:22:27.471 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 1.2 2024-09-22T03:22:27.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:27.610+0000 7f5d43a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:27.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:27.623+0000 7f5d43a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:27.918 INFO:teuthology.orchestra.run.smithi146.stderr:instructing pg(s) [1.2] on osd.2 to force-recovery; 2024-09-22T03:22:27.918 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:22:29.378 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:29.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:29.501+0000 7f3b44c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:29.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:29.507+0000 7f3b44c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:31.705 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:31.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:31.837+0000 7fa527d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:31.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:31.844+0000 7fa527d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:32.936 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:22:32.936 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:22:32.936 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:22:32.936 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:22:33.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.075+0000 7f8102d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.087+0000 7f8102d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.121 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:33.122 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:33.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:33.128 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:33.128 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:22:33.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.259+0000 7f9ef9117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.270+0000 7f9ef9117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:33.324 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:33.328 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:33.328 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:33.329 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:22:33.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.470+0000 7f4408d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.481+0000 7f4408d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:33.598 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:33.598 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:33.598 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:22:33.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.721+0000 7f6f5822c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:33.731+0000 7f6f5822c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:33.776 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:33.777 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:33.780 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:33.780 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:34.099 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:34.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:34.219+0000 7f031c42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:34.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:34.225+0000 7f031c42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:36.425 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:36.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:36.579+0000 7f956142c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:36.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:36.585+0000 7f956142c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:38.781 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:22:38.781 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:22:38.781 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:22:38.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:22:38.886 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:38.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:38.930+0000 7f629ef57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:38.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:38.941+0000 7f629ef57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.025+0000 7fccd106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.030+0000 7fccd106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.047 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:39.047 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:39.048 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:39.050 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:39.051 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:39.051 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:22:39.183 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.176+0000 7ff6e20fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.186+0000 7ff6e20fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.221 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:39.221 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:39.222 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:39.223 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:39.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:39.225 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:39.225 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:22:39.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.372+0000 7f95fc397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.381+0000 7f95fc397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:39.461 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:39.462 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:39.462 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:39.462 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:39.464 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:39.465 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:39.465 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:22:39.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.593+0000 7f9c10e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:39.605+0000 7f9c10e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:22:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:22:39.674 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:22:39.674 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:22:39.676 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:39.676 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:22:41.283 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:41.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:41.422+0000 7f100e4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:41.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:41.428+0000 7f100e4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:43.626 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:22:43.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:43.781+0000 7f1df9997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:43.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:43.787+0000 7f1df9997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:44.677 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:22:44.677 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:22:44.678 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 3, 1, 2] 2024-09-22T03:22:44.678 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:22:44.678 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:22:46.107 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:22:46.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:46.210+0000 7f98e9b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:46.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:22:46.215+0000 7f98e9b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:48.256 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:22:50.779 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:22:53.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:22:55.780 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 3, 2] 2024-09-22T03:22:55.780 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:22:55.780 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:22:55.780 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:22:55.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:22:55.783 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:22:55.783 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:22:55.896 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:22:55.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:22:55.900 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:23:00.901 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:23:00.997 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:23:01.018 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:01.018 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:23:06.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:23:06.116 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:23:06.118 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:06.119 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:23:08.126 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:23:08.229 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:23:08.229 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:23:10.230 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:10.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:10.366+0000 7f86196d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:10.376+0000 7f86196d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.120 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:23:11.267 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:23:11.267 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:23:11.267 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.267 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "replicated_request", 2024-09-22T03:23:11.267 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 321, 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "RepRequest": { 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "reqid": "client.4937.0:26601", 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "5.8", 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 146, 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "min_epoch": 145, 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "oid": "5:17a7fe84:::benchmark_data_smithi146_42560_object9662:head", 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "3" 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.268 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 166, 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.285327+0000" 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.285348+0000" 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.269 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.285430+0000" 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.285488+0000" 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.286077+0000" 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.287229+0000" 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.286446+0000" 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.286652+0000" 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.286828+0000" 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.276 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.277 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.279 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 167, 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.280 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.285984+0000" 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.285989+0000" 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.286471+0000" 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.282 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.286652+0000" 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.287429+0000" 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.332467+0000" 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.284 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.287838+0000" 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.288154+0000" 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.286 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.289075+0000" 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.288 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.289319+0000" 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.290583+0000" 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.332018+0000" 2024-09-22T03:23:11.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 183, 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.292 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.334905+0000" 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.334912+0000" 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.334977+0000" 2024-09-22T03:23:11.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.337995+0000" 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.338245+0000" 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.295 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.373488+0000" 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.338946+0000" 2024-09-22T03:23:11.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.339114+0000" 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.339265+0000" 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.339502+0000" 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.341791+0000" 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.368287+0000" 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 189, 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.362615+0000" 2024-09-22T03:23:11.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.362628+0000" 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.362753+0000" 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.362977+0000" 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.367941+0000" 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.396989+0000" 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.369092+0000" 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.308 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.369286+0000" 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.369447+0000" 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.312 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.373733+0000" 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.375843+0000" 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.396720+0000" 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 201, 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.458264+0000" 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.316 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.458270+0000" 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.458351+0000" 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.458398+0000" 2024-09-22T03:23:11.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.458567+0000" 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.476304+0000" 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.458957+0000" 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.320 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.459136+0000" 2024-09-22T03:23:11.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.322 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.459288+0000" 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.323 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.324 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.459530+0000" 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.460713+0000" 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.475914+0000" 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 210, 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.327 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.494985+0000" 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.494991+0000" 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.495074+0000" 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.328 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.495121+0000" 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.329 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.495279+0000" 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.330 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.534584+0000" 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.495621+0000" 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.495801+0000" 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.332 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.495956+0000" 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.333 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.334 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.496184+0000" 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.335 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.497480+0000" 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.534141+0000" 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.337 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 211, 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.496720+0000" 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.496730+0000" 2024-09-22T03:23:11.338 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.496953+0000" 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.497033+0000" 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.498274+0000" 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.340 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.536572+0000" 2024-09-22T03:23:11.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.502812+0000" 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.342 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.503040+0000" 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.343 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.503200+0000" 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.344 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.503438+0000" 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.346 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.504772+0000" 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.347 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.536192+0000" 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 216, 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.523120+0000" 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.349 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.523133+0000" 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.523264+0000" 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.350 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.523353+0000" 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.351 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.523481+0000" 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.554217+0000" 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.523696+0000" 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.353 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.523883+0000" 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.354 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.524052+0000" 2024-09-22T03:23:11.355 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.356 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.357 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.524311+0000" 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.525281+0000" 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.359 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.553775+0000" 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.360 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 221, 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.556485+0000" 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.556496+0000" 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.556629+0000" 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.362 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.556725+0000" 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.556933+0000" 2024-09-22T03:23:11.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.364 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.610486+0000" 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.557299+0000" 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.365 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.557473+0000" 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.366 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.557627+0000" 2024-09-22T03:23:11.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.368 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.369 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.557894+0000" 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.370 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.559217+0000" 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.371 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.610226+0000" 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 240, 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.719299+0000" 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.719307+0000" 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.373 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.719394+0000" 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.374 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.719449+0000" 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.719615+0000" 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.375 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.747375+0000" 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.376 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.719844+0000" 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.719989+0000" 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.720141+0000" 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.720376+0000" 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.381 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.721676+0000" 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.382 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.746921+0000" 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.383 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 241, 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.721721+0000" 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.721728+0000" 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.384 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.721832+0000" 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.721879+0000" 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.386 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.732714+0000" 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.387 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.754885+0000" 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.733021+0000" 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.733136+0000" 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.389 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.733248+0000" 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.391 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.733715+0000" 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.735358+0000" 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.393 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.754621+0000" 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.394 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 242, 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.395 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.725939+0000" 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.725946+0000" 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.396 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.726042+0000" 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.397 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.726091+0000" 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.732756+0000" 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.398 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.762690+0000" 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.733496+0000" 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.733809+0000" 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.401 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.734114+0000" 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.402 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.403 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.734342+0000" 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.404 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.739956+0000" 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.405 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.762421+0000" 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.406 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 246, 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.766728+0000" 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.766735+0000" 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.407 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.766829+0000" 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.408 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.766884+0000" 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.767412+0000" 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.409 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.786016+0000" 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.767616+0000" 2024-09-22T03:23:11.410 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.411 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.767716+0000" 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.412 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.767822+0000" 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.767966+0000" 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.768920+0000" 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.785592+0000" 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.417 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 249, 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.778822+0000" 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.778826+0000" 2024-09-22T03:23:11.418 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.778945+0000" 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.419 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.778986+0000" 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.420 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.779086+0000" 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.421 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.804693+0000" 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.779288+0000" 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.422 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.779394+0000" 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.423 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.779481+0000" 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.424 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.425 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.779606+0000" 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.426 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.780557+0000" 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.804220+0000" 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.428 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 256, 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.823716+0000" 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.429 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.823722+0000" 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.823804+0000" 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.430 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.823858+0000" 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.431 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.824038+0000" 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.432 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.853447+0000" 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.824384+0000" 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.433 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.824557+0000" 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.434 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.824698+0000" 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.435 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.436 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.824981+0000" 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.826290+0000" 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.439 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.853195+0000" 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.440 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 265, 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881188+0000" 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881195+0000" 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881274+0000" 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.442 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881328+0000" 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881504+0000" 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.443 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.900008+0000" 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.444 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881709+0000" 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.445 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881817+0000" 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.446 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.881913+0000" 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.447 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.448 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.882091+0000" 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.449 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.883128+0000" 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.450 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:10.899712+0000" 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.451 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 278, 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002087+0000" 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.452 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002093+0000" 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002172+0000" 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.453 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002221+0000" 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.454 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002401+0000" 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.455 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.022754+0000" 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002612+0000" 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.456 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002704+0000" 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.457 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002790+0000" 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.458 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.459 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.002919+0000" 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.461 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.003878+0000" 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.462 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.022332+0000" 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 297, 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.103947+0000" 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.103954+0000" 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.464 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.104036+0000" 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.465 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.104084+0000" 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.466 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.107007+0000" 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.467 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.129877+0000" 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.107517+0000" 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.107875+0000" 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.107993+0000" 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.470 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.108146+0000" 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.472 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.112674+0000" 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.473 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:23:11.129463+0000" 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:23:11.474 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:23:11.475 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:23:12.551 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:12.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:12.672+0000 7f6b8bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:12.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:12.677+0000 7f6b8bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.796 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:14.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:14.927+0000 7fe149afa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:14.933+0000 7fe149afa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:16.477 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 2, 1] 2024-09-22T03:23:16.477 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:23:16.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:16.626+0000 7f185e517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:16.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:16.642+0000 7f185e517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:17.155 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:23:17.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:23:17.243 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:17.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:17.309+0000 7f9a40e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:17.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:17.320+0000 7f9a40e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:17.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:17.391+0000 7f1434b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:17.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:17.400+0000 7f1434b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:18.159 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:23:18.177 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:23:18.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:18.312+0000 7f8e33e18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:18.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:18.324+0000 7f8e33e18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.163 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:23:19.185 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:23:19.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:19.309+0000 7f79d3d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:19.323+0000 7f79d3d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.712 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:19.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:19.865+0000 7fd611b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:19.872+0000 7fd611b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:20.093 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:23:20.115 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:23:20.115 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:20.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:20.264+0000 7f712c86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:20.274+0000 7f712c86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:20.572 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:23:20.572 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:23:20.588 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-22T03:23:20.093141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316211787,"num_objects":4821,"num_object_clones":0,"num_object_copies":14463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":29196,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5107,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29144,"ondisk_log_size":29144,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":262064,"kb_used_data":68012,"kb_used_omap":0,"kb_used_meta":193984,"kb_avail":377225296,"statfs":{"total":386547056640,"available":386278703104,"internally_reserved":0,"allocated":69644288,"data_stored":1035735351,"data_compressed":49091468,"data_compressed_allocated":64618496,"data_compressed_original":1033650176,"omap_allocated":0,"internal_metadata":198639616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-107479040,"num_objects":-1640,"num_object_clones":0,"num_object_copies":-4920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1640,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.713634"},"pg_stats":[{"pgid":"5.f","version":"149'1743","reported_seq":1873,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.562656+0000","last_change":"2024-09-22T03:22:52.122129+0000","last_active":"2024-09-22T03:23:17.562656+0000","last_peered":"2024-09-22T03:23:17.562656+0000","last_clean":"2024-09-22T03:23:17.562656+0000","last_became_active":"2024-09-22T03:22:52.119107+0000","last_became_peered":"2024-09-22T03:22:52.119107+0000","last_unstale":"2024-09-22T03:23:17.562656+0000","last_undegraded":"2024-09-22T03:23:17.562656+0000","last_fullsized":"2024-09-22T03:23:17.562656+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1743,"log_dups_size":0,"ondisk_log_size":1743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1744,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"150'1842","reported_seq":2601,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.180621+0000","last_change":"2024-09-22T03:22:35.180867+0000","last_active":"2024-09-22T03:23:19.180621+0000","last_peered":"2024-09-22T03:23:19.180621+0000","last_clean":"2024-09-22T03:23:19.180621+0000","last_became_active":"2024-09-22T03:22:35.167049+0000","last_became_peered":"2024-09-22T03:22:35.167049+0000","last_unstale":"2024-09-22T03:23:19.180621+0000","last_undegraded":"2024-09-22T03:23:19.180621+0000","last_fullsized":"2024-09-22T03:23:19.180621+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1842,"log_dups_size":0,"ondisk_log_size":1842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":262,"num_object_clones":0,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"150'1855","reported_seq":2598,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.190700+0000","last_change":"2024-09-22T03:22:59.147258+0000","last_active":"2024-09-22T03:23:19.190700+0000","last_peered":"2024-09-22T03:23:19.190700+0000","last_clean":"2024-09-22T03:23:19.190700+0000","last_became_active":"2024-09-22T03:22:59.144464+0000","last_became_peered":"2024-09-22T03:22:59.144464+0000","last_unstale":"2024-09-22T03:23:19.190700+0000","last_undegraded":"2024-09-22T03:23:19.190700+0000","last_fullsized":"2024-09-22T03:23:19.190700+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1855,"log_dups_size":0,"ondisk_log_size":1855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629208,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1856,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":196632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"149'1861","reported_seq":2603,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.033315+0000","last_change":"2024-09-22T03:23:10.167406+0000","last_active":"2024-09-22T03:23:18.033315+0000","last_peered":"2024-09-22T03:23:18.033315+0000","last_clean":"2024-09-22T03:23:18.033315+0000","last_became_active":"2024-09-22T03:23:10.163693+0000","last_became_peered":"2024-09-22T03:23:10.163693+0000","last_unstale":"2024-09-22T03:23:18.033315+0000","last_undegraded":"2024-09-22T03:23:18.033315+0000","last_fullsized":"2024-09-22T03:23:18.033315+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1861,"log_dups_size":0,"ondisk_log_size":1861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":284,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1862,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"150'1814","reported_seq":1985,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.184187+0000","last_change":"2024-09-22T03:23:10.226285+0000","last_active":"2024-09-22T03:23:19.184187+0000","last_peered":"2024-09-22T03:23:19.184187+0000","last_clean":"2024-09-22T03:23:19.184187+0000","last_became_active":"2024-09-22T03:23:10.217776+0000","last_became_peered":"2024-09-22T03:23:10.217776+0000","last_unstale":"2024-09-22T03:23:19.184187+0000","last_undegraded":"2024-09-22T03:23:19.184187+0000","last_fullsized":"2024-09-22T03:23:19.184187+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1814,"log_dups_size":0,"ondisk_log_size":1814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1814,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"149'1782","reported_seq":1909,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.517070+0000","last_change":"2024-09-22T03:22:49.189015+0000","last_active":"2024-09-22T03:23:17.517070+0000","last_peered":"2024-09-22T03:23:17.517070+0000","last_clean":"2024-09-22T03:23:17.517070+0000","last_became_active":"2024-09-22T03:22:49.185334+0000","last_became_peered":"2024-09-22T03:22:49.185334+0000","last_unstale":"2024-09-22T03:23:17.517070+0000","last_undegraded":"2024-09-22T03:23:17.517070+0000","last_fullsized":"2024-09-22T03:23:17.517070+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1782,"log_dups_size":0,"ondisk_log_size":1782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"150'1841","reported_seq":2539,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.862892+0000","last_change":"2024-09-22T03:23:10.232225+0000","last_active":"2024-09-22T03:23:18.862892+0000","last_peered":"2024-09-22T03:23:18.862892+0000","last_clean":"2024-09-22T03:23:18.862892+0000","last_became_active":"2024-09-22T03:23:10.223267+0000","last_became_peered":"2024-09-22T03:23:10.223267+0000","last_unstale":"2024-09-22T03:23:18.862892+0000","last_undegraded":"2024-09-22T03:23:18.862892+0000","last_fullsized":"2024-09-22T03:23:18.862892+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1841,"log_dups_size":0,"ondisk_log_size":1841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529728,"num_objects":298,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"149'1831","reported_seq":4684,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.998258+0000","last_change":"2024-09-22T03:23:13.163518+0000","last_active":"2024-09-22T03:23:17.998258+0000","last_peered":"2024-09-22T03:23:17.998258+0000","last_clean":"2024-09-22T03:23:17.998258+0000","last_became_active":"2024-09-22T03:23:13.157048+0000","last_became_peered":"2024-09-22T03:23:13.157048+0000","last_unstale":"2024-09-22T03:23:17.998258+0000","last_undegraded":"2024-09-22T03:23:17.998258+0000","last_fullsized":"2024-09-22T03:23:17.998258+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1831,"log_dups_size":0,"ondisk_log_size":1831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1832,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":19726336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"149'1730","reported_seq":2421,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.546038+0000","last_change":"2024-09-22T03:23:10.319816+0000","last_active":"2024-09-22T03:23:17.546038+0000","last_peered":"2024-09-22T03:23:17.546038+0000","last_clean":"2024-09-22T03:23:17.546038+0000","last_became_active":"2024-09-22T03:23:10.216069+0000","last_became_peered":"2024-09-22T03:23:10.216069+0000","last_unstale":"2024-09-22T03:23:17.546038+0000","last_undegraded":"2024-09-22T03:23:17.546038+0000","last_fullsized":"2024-09-22T03:23:17.546038+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1730,"log_dups_size":0,"ondisk_log_size":1730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1731,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":221,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.169670+0000","last_change":"2024-09-22T03:22:24.509584+0000","last_active":"2024-09-22T03:23:17.169670+0000","last_peered":"2024-09-22T03:23:17.169670+0000","last_clean":"2024-09-22T03:23:17.169670+0000","last_became_active":"2024-09-22T03:22:23.564776+0000","last_became_peered":"2024-09-22T03:22:23.564776+0000","last_unstale":"2024-09-22T03:23:17.169670+0000","last_undegraded":"2024-09-22T03:23:17.169670+0000","last_fullsized":"2024-09-22T03:23:17.169670+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"149'1841","reported_seq":2557,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.581445+0000","last_change":"2024-09-22T03:23:12.179998+0000","last_active":"2024-09-22T03:23:17.581445+0000","last_peered":"2024-09-22T03:23:17.581445+0000","last_clean":"2024-09-22T03:23:17.581445+0000","last_became_active":"2024-09-22T03:23:10.163916+0000","last_became_peered":"2024-09-22T03:23:10.163916+0000","last_unstale":"2024-09-22T03:23:17.581445+0000","last_undegraded":"2024-09-22T03:23:17.581445+0000","last_fullsized":"2024-09-22T03:23:17.581445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1841,"log_dups_size":0,"ondisk_log_size":1841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":351,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.185575+0000","last_change":"2024-09-22T03:23:11.108176+0000","last_active":"2024-09-22T03:23:19.185575+0000","last_peered":"2024-09-22T03:23:19.185575+0000","last_clean":"2024-09-22T03:23:19.185575+0000","last_became_active":"2024-09-22T03:23:10.220262+0000","last_became_peered":"2024-09-22T03:23:10.220262+0000","last_unstale":"2024-09-22T03:23:19.185575+0000","last_undegraded":"2024-09-22T03:23:19.185575+0000","last_fullsized":"2024-09-22T03:23:19.185575+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"149'1857","reported_seq":2400,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.329248+0000","last_change":"2024-09-22T03:22:47.181742+0000","last_active":"2024-09-22T03:23:17.329248+0000","last_peered":"2024-09-22T03:23:17.329248+0000","last_clean":"2024-09-22T03:23:17.329248+0000","last_became_active":"2024-09-22T03:22:47.177767+0000","last_became_peered":"2024-09-22T03:22:47.177767+0000","last_unstale":"2024-09-22T03:23:17.329248+0000","last_undegraded":"2024-09-22T03:23:17.329248+0000","last_fullsized":"2024-09-22T03:23:17.329248+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1857,"log_dups_size":0,"ondisk_log_size":1857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757952,"num_objects":332,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":418,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.176641+0000","last_change":"2024-09-22T03:23:11.125366+0000","last_active":"2024-09-22T03:23:17.176641+0000","last_peered":"2024-09-22T03:23:17.176641+0000","last_clean":"2024-09-22T03:23:17.176641+0000","last_became_active":"2024-09-22T03:23:10.182182+0000","last_became_peered":"2024-09-22T03:23:10.182182+0000","last_unstale":"2024-09-22T03:23:17.176641+0000","last_undegraded":"2024-09-22T03:23:17.176641+0000","last_fullsized":"2024-09-22T03:23:17.176641+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"150'1784","reported_seq":2506,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.185744+0000","last_change":"2024-09-22T03:23:10.349008+0000","last_active":"2024-09-22T03:23:19.185744+0000","last_peered":"2024-09-22T03:23:19.185744+0000","last_clean":"2024-09-22T03:23:19.185744+0000","last_became_active":"2024-09-22T03:23:10.218846+0000","last_became_peered":"2024-09-22T03:23:10.218846+0000","last_unstale":"2024-09-22T03:23:19.185744+0000","last_undegraded":"2024-09-22T03:23:19.185744+0000","last_fullsized":"2024-09-22T03:23:19.185744+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1784,"log_dups_size":0,"ondisk_log_size":1784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1784,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":304,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.169118+0000","last_change":"2024-09-22T03:23:11.112920+0000","last_active":"2024-09-22T03:23:17.169118+0000","last_peered":"2024-09-22T03:23:17.169118+0000","last_clean":"2024-09-22T03:23:17.169118+0000","last_became_active":"2024-09-22T03:23:10.215616+0000","last_became_peered":"2024-09-22T03:23:10.215616+0000","last_unstale":"2024-09-22T03:23:17.169118+0000","last_undegraded":"2024-09-22T03:23:17.169118+0000","last_fullsized":"2024-09-22T03:23:17.169118+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":471,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.185188+0000","last_change":"2024-09-22T03:23:10.228555+0000","last_active":"2024-09-22T03:23:19.185188+0000","last_peered":"2024-09-22T03:23:19.185188+0000","last_clean":"2024-09-22T03:23:19.185188+0000","last_became_active":"2024-09-22T03:23:10.223350+0000","last_became_peered":"2024-09-22T03:23:10.223350+0000","last_unstale":"2024-09-22T03:23:19.185188+0000","last_undegraded":"2024-09-22T03:23:19.185188+0000","last_fullsized":"2024-09-22T03:23:19.185188+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"150'1907","reported_seq":2668,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.184870+0000","last_change":"2024-09-22T03:23:10.421117+0000","last_active":"2024-09-22T03:23:19.184870+0000","last_peered":"2024-09-22T03:23:19.184870+0000","last_clean":"2024-09-22T03:23:19.184870+0000","last_became_active":"2024-09-22T03:23:10.242427+0000","last_became_peered":"2024-09-22T03:23:10.242427+0000","last_unstale":"2024-09-22T03:23:19.184870+0000","last_undegraded":"2024-09-22T03:23:19.184870+0000","last_fullsized":"2024-09-22T03:23:19.184870+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1907,"log_dups_size":0,"ondisk_log_size":1907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1908,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":159,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.191329+0000","last_change":"2024-09-22T03:23:11.117757+0000","last_active":"2024-09-22T03:23:18.191329+0000","last_peered":"2024-09-22T03:23:18.191329+0000","last_clean":"2024-09-22T03:23:18.191329+0000","last_became_active":"2024-09-22T03:23:10.226835+0000","last_became_peered":"2024-09-22T03:23:10.226835+0000","last_unstale":"2024-09-22T03:23:18.191329+0000","last_undegraded":"2024-09-22T03:23:18.191329+0000","last_fullsized":"2024-09-22T03:23:18.191329+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"149'1879","reported_seq":2616,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.045198+0000","last_change":"2024-09-22T03:22:43.211209+0000","last_active":"2024-09-22T03:23:18.045198+0000","last_peered":"2024-09-22T03:23:18.045198+0000","last_clean":"2024-09-22T03:23:18.045198+0000","last_became_active":"2024-09-22T03:22:43.201439+0000","last_became_peered":"2024-09-22T03:22:43.201439+0000","last_unstale":"2024-09-22T03:23:18.045198+0000","last_undegraded":"2024-09-22T03:23:18.045198+0000","last_fullsized":"2024-09-22T03:23:18.045198+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1879,"log_dups_size":0,"ondisk_log_size":1879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1880,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":402,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.173357+0000","last_change":"2024-09-22T03:22:24.447081+0000","last_active":"2024-09-22T03:23:17.173357+0000","last_peered":"2024-09-22T03:23:17.173357+0000","last_clean":"2024-09-22T03:23:17.173357+0000","last_became_active":"2024-09-22T03:22:23.585123+0000","last_became_peered":"2024-09-22T03:22:23.585123+0000","last_unstale":"2024-09-22T03:23:17.173357+0000","last_undegraded":"2024-09-22T03:23:17.173357+0000","last_fullsized":"2024-09-22T03:23:17.173357+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"150'1668","reported_seq":1837,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.184027+0000","last_change":"2024-09-22T03:23:10.274356+0000","last_active":"2024-09-22T03:23:19.184027+0000","last_peered":"2024-09-22T03:23:19.184027+0000","last_clean":"2024-09-22T03:23:19.184027+0000","last_became_active":"2024-09-22T03:23:10.215382+0000","last_became_peered":"2024-09-22T03:23:10.215382+0000","last_unstale":"2024-09-22T03:23:19.184027+0000","last_undegraded":"2024-09-22T03:23:19.184027+0000","last_fullsized":"2024-09-22T03:23:19.184027+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1668,"log_dups_size":0,"ondisk_log_size":1668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":295,"num_object_clones":0,"num_object_copies":885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":277,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.195393+0000","last_change":"2024-09-22T03:23:11.109171+0000","last_active":"2024-09-22T03:23:18.195393+0000","last_peered":"2024-09-22T03:23:18.195393+0000","last_clean":"2024-09-22T03:23:18.195393+0000","last_became_active":"2024-09-22T03:23:10.194313+0000","last_became_peered":"2024-09-22T03:23:10.194313+0000","last_unstale":"2024-09-22T03:23:18.195393+0000","last_undegraded":"2024-09-22T03:23:18.195393+0000","last_fullsized":"2024-09-22T03:23:18.195393+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"149'1809","reported_seq":2420,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.450391+0000","last_change":"2024-09-22T03:23:10.772811+0000","last_active":"2024-09-22T03:23:17.450391+0000","last_peered":"2024-09-22T03:23:17.450391+0000","last_clean":"2024-09-22T03:23:17.450391+0000","last_became_active":"2024-09-22T03:23:10.188905+0000","last_became_peered":"2024-09-22T03:23:10.188905+0000","last_unstale":"2024-09-22T03:23:17.450391+0000","last_undegraded":"2024-09-22T03:23:17.450391+0000","last_fullsized":"2024-09-22T03:23:17.450391+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1809,"log_dups_size":0,"ondisk_log_size":1809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":320,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":232,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.196846+0000","last_change":"2024-09-22T03:23:11.109551+0000","last_active":"2024-09-22T03:23:18.196846+0000","last_peered":"2024-09-22T03:23:18.196846+0000","last_clean":"2024-09-22T03:23:18.196846+0000","last_became_active":"2024-09-22T03:23:10.218679+0000","last_became_peered":"2024-09-22T03:23:10.218679+0000","last_unstale":"2024-09-22T03:23:18.196846+0000","last_undegraded":"2024-09-22T03:23:18.196846+0000","last_fullsized":"2024-09-22T03:23:18.196846+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":315621400,"num_objects":4817,"num_object_clones":0,"num_object_copies":14451,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":29057,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5107,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62111744,"data_stored":993591368,"data_compressed":47181032,"data_compressed_allocated":62099456,"data_compressed_original":993591296,"omap_allocated":0,"internal_metadata":0},"log_size":29044,"ondisk_log_size":29044,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676598967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66908,"kb_used_data":16652,"kb_used_omap":0,"kb_used_meta":50240,"kb_avail":94304932,"statfs":{"total":96636764160,"available":96568250368,"internally_reserved":0,"allocated":17051648,"data_stored":253476743,"data_compressed":12012320,"data_compressed_allocated":15810560,"data_compressed_original":252968960,"omap_allocated":0,"internal_metadata":51445760},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":119,"seq":511101108279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63548,"kb_used_data":19116,"kb_used_omap":0,"kb_used_meta":44416,"kb_avail":94308292,"statfs":{"total":96636764160,"available":96571691008,"internally_reserved":0,"allocated":19574784,"data_stored":293454247,"data_compressed":13912604,"data_compressed_allocated":18313216,"data_compressed_original":292929536,"omap_allocated":0,"internal_metadata":45481984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":144,"seq":618475290633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52548,"kb_used_data":13552,"kb_used_omap":0,"kb_used_meta":38976,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":13877248,"data_stored":202359183,"data_compressed":9586924,"data_compressed_allocated":12619776,"data_compressed_original":201834496,"omap_allocated":0,"internal_metadata":39911424},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79060,"kb_used_data":18692,"kb_used_omap":0,"kb_used_meta":60352,"kb_avail":94292780,"statfs":{"total":96636764160,"available":96555806720,"internally_reserved":0,"allocated":19140608,"data_stored":286445178,"data_compressed":13579620,"data_compressed_allocated":17874944,"data_compressed_original":285917184,"omap_allocated":0,"internal_metadata":61800448},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17154048,"data_stored":274399256,"data_compressed":13029944,"data_compressed_allocated":17149952,"data_compressed_original":274399232,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11898880,"data_stored":190382080,"data_compressed":9040360,"data_compressed_allocated":11898880,"data_compressed_original":190382080,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18141184,"data_stored":290193432,"data_compressed":13779936,"data_compressed_allocated":18137088,"data_compressed_original":290193408,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14917632,"data_stored":238616600,"data_compressed":11330792,"data_compressed_allocated":14913536,"data_compressed_original":238616576,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:20.590 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:20.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:20.713+0000 7f04a7797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:20.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:20.724+0000 7f04a7797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:21.001 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:23:21.001 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:23:21.019 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-22T03:23:20.093141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316211787,"num_objects":4821,"num_object_clones":0,"num_object_copies":14463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":29196,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5107,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29144,"ondisk_log_size":29144,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":262064,"kb_used_data":68012,"kb_used_omap":0,"kb_used_meta":193984,"kb_avail":377225296,"statfs":{"total":386547056640,"available":386278703104,"internally_reserved":0,"allocated":69644288,"data_stored":1035735351,"data_compressed":49091468,"data_compressed_allocated":64618496,"data_compressed_original":1033650176,"omap_allocated":0,"internal_metadata":198639616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-107479040,"num_objects":-1640,"num_object_clones":0,"num_object_copies":-4920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1640,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.713634"},"pg_stats":[{"pgid":"5.f","version":"149'1743","reported_seq":1873,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.562656+0000","last_change":"2024-09-22T03:22:52.122129+0000","last_active":"2024-09-22T03:23:17.562656+0000","last_peered":"2024-09-22T03:23:17.562656+0000","last_clean":"2024-09-22T03:23:17.562656+0000","last_became_active":"2024-09-22T03:22:52.119107+0000","last_became_peered":"2024-09-22T03:22:52.119107+0000","last_unstale":"2024-09-22T03:23:17.562656+0000","last_undegraded":"2024-09-22T03:23:17.562656+0000","last_fullsized":"2024-09-22T03:23:17.562656+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1743,"log_dups_size":0,"ondisk_log_size":1743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1744,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"150'1842","reported_seq":2601,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.180621+0000","last_change":"2024-09-22T03:22:35.180867+0000","last_active":"2024-09-22T03:23:19.180621+0000","last_peered":"2024-09-22T03:23:19.180621+0000","last_clean":"2024-09-22T03:23:19.180621+0000","last_became_active":"2024-09-22T03:22:35.167049+0000","last_became_peered":"2024-09-22T03:22:35.167049+0000","last_unstale":"2024-09-22T03:23:19.180621+0000","last_undegraded":"2024-09-22T03:23:19.180621+0000","last_fullsized":"2024-09-22T03:23:19.180621+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1842,"log_dups_size":0,"ondisk_log_size":1842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":262,"num_object_clones":0,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"150'1855","reported_seq":2598,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.190700+0000","last_change":"2024-09-22T03:22:59.147258+0000","last_active":"2024-09-22T03:23:19.190700+0000","last_peered":"2024-09-22T03:23:19.190700+0000","last_clean":"2024-09-22T03:23:19.190700+0000","last_became_active":"2024-09-22T03:22:59.144464+0000","last_became_peered":"2024-09-22T03:22:59.144464+0000","last_unstale":"2024-09-22T03:23:19.190700+0000","last_undegraded":"2024-09-22T03:23:19.190700+0000","last_fullsized":"2024-09-22T03:23:19.190700+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1855,"log_dups_size":0,"ondisk_log_size":1855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629208,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1856,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":196632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"149'1861","reported_seq":2603,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.033315+0000","last_change":"2024-09-22T03:23:10.167406+0000","last_active":"2024-09-22T03:23:18.033315+0000","last_peered":"2024-09-22T03:23:18.033315+0000","last_clean":"2024-09-22T03:23:18.033315+0000","last_became_active":"2024-09-22T03:23:10.163693+0000","last_became_peered":"2024-09-22T03:23:10.163693+0000","last_unstale":"2024-09-22T03:23:18.033315+0000","last_undegraded":"2024-09-22T03:23:18.033315+0000","last_fullsized":"2024-09-22T03:23:18.033315+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1861,"log_dups_size":0,"ondisk_log_size":1861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":284,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1862,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"150'1814","reported_seq":1985,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.184187+0000","last_change":"2024-09-22T03:23:10.226285+0000","last_active":"2024-09-22T03:23:19.184187+0000","last_peered":"2024-09-22T03:23:19.184187+0000","last_clean":"2024-09-22T03:23:19.184187+0000","last_became_active":"2024-09-22T03:23:10.217776+0000","last_became_peered":"2024-09-22T03:23:10.217776+0000","last_unstale":"2024-09-22T03:23:19.184187+0000","last_undegraded":"2024-09-22T03:23:19.184187+0000","last_fullsized":"2024-09-22T03:23:19.184187+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1814,"log_dups_size":0,"ondisk_log_size":1814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1814,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"149'1782","reported_seq":1909,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.517070+0000","last_change":"2024-09-22T03:22:49.189015+0000","last_active":"2024-09-22T03:23:17.517070+0000","last_peered":"2024-09-22T03:23:17.517070+0000","last_clean":"2024-09-22T03:23:17.517070+0000","last_became_active":"2024-09-22T03:22:49.185334+0000","last_became_peered":"2024-09-22T03:22:49.185334+0000","last_unstale":"2024-09-22T03:23:17.517070+0000","last_undegraded":"2024-09-22T03:23:17.517070+0000","last_fullsized":"2024-09-22T03:23:17.517070+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1782,"log_dups_size":0,"ondisk_log_size":1782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"150'1841","reported_seq":2539,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.862892+0000","last_change":"2024-09-22T03:23:10.232225+0000","last_active":"2024-09-22T03:23:18.862892+0000","last_peered":"2024-09-22T03:23:18.862892+0000","last_clean":"2024-09-22T03:23:18.862892+0000","last_became_active":"2024-09-22T03:23:10.223267+0000","last_became_peered":"2024-09-22T03:23:10.223267+0000","last_unstale":"2024-09-22T03:23:18.862892+0000","last_undegraded":"2024-09-22T03:23:18.862892+0000","last_fullsized":"2024-09-22T03:23:18.862892+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1841,"log_dups_size":0,"ondisk_log_size":1841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529728,"num_objects":298,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"149'1831","reported_seq":4684,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.998258+0000","last_change":"2024-09-22T03:23:13.163518+0000","last_active":"2024-09-22T03:23:17.998258+0000","last_peered":"2024-09-22T03:23:17.998258+0000","last_clean":"2024-09-22T03:23:17.998258+0000","last_became_active":"2024-09-22T03:23:13.157048+0000","last_became_peered":"2024-09-22T03:23:13.157048+0000","last_unstale":"2024-09-22T03:23:17.998258+0000","last_undegraded":"2024-09-22T03:23:17.998258+0000","last_fullsized":"2024-09-22T03:23:17.998258+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1831,"log_dups_size":0,"ondisk_log_size":1831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1832,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":19726336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"149'1730","reported_seq":2421,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.546038+0000","last_change":"2024-09-22T03:23:10.319816+0000","last_active":"2024-09-22T03:23:17.546038+0000","last_peered":"2024-09-22T03:23:17.546038+0000","last_clean":"2024-09-22T03:23:17.546038+0000","last_became_active":"2024-09-22T03:23:10.216069+0000","last_became_peered":"2024-09-22T03:23:10.216069+0000","last_unstale":"2024-09-22T03:23:17.546038+0000","last_undegraded":"2024-09-22T03:23:17.546038+0000","last_fullsized":"2024-09-22T03:23:17.546038+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1730,"log_dups_size":0,"ondisk_log_size":1730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1731,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":221,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.169670+0000","last_change":"2024-09-22T03:22:24.509584+0000","last_active":"2024-09-22T03:23:17.169670+0000","last_peered":"2024-09-22T03:23:17.169670+0000","last_clean":"2024-09-22T03:23:17.169670+0000","last_became_active":"2024-09-22T03:22:23.564776+0000","last_became_peered":"2024-09-22T03:22:23.564776+0000","last_unstale":"2024-09-22T03:23:17.169670+0000","last_undegraded":"2024-09-22T03:23:17.169670+0000","last_fullsized":"2024-09-22T03:23:17.169670+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"149'1841","reported_seq":2557,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.581445+0000","last_change":"2024-09-22T03:23:12.179998+0000","last_active":"2024-09-22T03:23:17.581445+0000","last_peered":"2024-09-22T03:23:17.581445+0000","last_clean":"2024-09-22T03:23:17.581445+0000","last_became_active":"2024-09-22T03:23:10.163916+0000","last_became_peered":"2024-09-22T03:23:10.163916+0000","last_unstale":"2024-09-22T03:23:17.581445+0000","last_undegraded":"2024-09-22T03:23:17.581445+0000","last_fullsized":"2024-09-22T03:23:17.581445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1841,"log_dups_size":0,"ondisk_log_size":1841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":351,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.185575+0000","last_change":"2024-09-22T03:23:11.108176+0000","last_active":"2024-09-22T03:23:19.185575+0000","last_peered":"2024-09-22T03:23:19.185575+0000","last_clean":"2024-09-22T03:23:19.185575+0000","last_became_active":"2024-09-22T03:23:10.220262+0000","last_became_peered":"2024-09-22T03:23:10.220262+0000","last_unstale":"2024-09-22T03:23:19.185575+0000","last_undegraded":"2024-09-22T03:23:19.185575+0000","last_fullsized":"2024-09-22T03:23:19.185575+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"149'1857","reported_seq":2400,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.329248+0000","last_change":"2024-09-22T03:22:47.181742+0000","last_active":"2024-09-22T03:23:17.329248+0000","last_peered":"2024-09-22T03:23:17.329248+0000","last_clean":"2024-09-22T03:23:17.329248+0000","last_became_active":"2024-09-22T03:22:47.177767+0000","last_became_peered":"2024-09-22T03:22:47.177767+0000","last_unstale":"2024-09-22T03:23:17.329248+0000","last_undegraded":"2024-09-22T03:23:17.329248+0000","last_fullsized":"2024-09-22T03:23:17.329248+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1857,"log_dups_size":0,"ondisk_log_size":1857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757952,"num_objects":332,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":418,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.176641+0000","last_change":"2024-09-22T03:23:11.125366+0000","last_active":"2024-09-22T03:23:17.176641+0000","last_peered":"2024-09-22T03:23:17.176641+0000","last_clean":"2024-09-22T03:23:17.176641+0000","last_became_active":"2024-09-22T03:23:10.182182+0000","last_became_peered":"2024-09-22T03:23:10.182182+0000","last_unstale":"2024-09-22T03:23:17.176641+0000","last_undegraded":"2024-09-22T03:23:17.176641+0000","last_fullsized":"2024-09-22T03:23:17.176641+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"150'1784","reported_seq":2506,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.185744+0000","last_change":"2024-09-22T03:23:10.349008+0000","last_active":"2024-09-22T03:23:19.185744+0000","last_peered":"2024-09-22T03:23:19.185744+0000","last_clean":"2024-09-22T03:23:19.185744+0000","last_became_active":"2024-09-22T03:23:10.218846+0000","last_became_peered":"2024-09-22T03:23:10.218846+0000","last_unstale":"2024-09-22T03:23:19.185744+0000","last_undegraded":"2024-09-22T03:23:19.185744+0000","last_fullsized":"2024-09-22T03:23:19.185744+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1784,"log_dups_size":0,"ondisk_log_size":1784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1784,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":304,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.169118+0000","last_change":"2024-09-22T03:23:11.112920+0000","last_active":"2024-09-22T03:23:17.169118+0000","last_peered":"2024-09-22T03:23:17.169118+0000","last_clean":"2024-09-22T03:23:17.169118+0000","last_became_active":"2024-09-22T03:23:10.215616+0000","last_became_peered":"2024-09-22T03:23:10.215616+0000","last_unstale":"2024-09-22T03:23:17.169118+0000","last_undegraded":"2024-09-22T03:23:17.169118+0000","last_fullsized":"2024-09-22T03:23:17.169118+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":471,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.185188+0000","last_change":"2024-09-22T03:23:10.228555+0000","last_active":"2024-09-22T03:23:19.185188+0000","last_peered":"2024-09-22T03:23:19.185188+0000","last_clean":"2024-09-22T03:23:19.185188+0000","last_became_active":"2024-09-22T03:23:10.223350+0000","last_became_peered":"2024-09-22T03:23:10.223350+0000","last_unstale":"2024-09-22T03:23:19.185188+0000","last_undegraded":"2024-09-22T03:23:19.185188+0000","last_fullsized":"2024-09-22T03:23:19.185188+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"150'1907","reported_seq":2668,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.184870+0000","last_change":"2024-09-22T03:23:10.421117+0000","last_active":"2024-09-22T03:23:19.184870+0000","last_peered":"2024-09-22T03:23:19.184870+0000","last_clean":"2024-09-22T03:23:19.184870+0000","last_became_active":"2024-09-22T03:23:10.242427+0000","last_became_peered":"2024-09-22T03:23:10.242427+0000","last_unstale":"2024-09-22T03:23:19.184870+0000","last_undegraded":"2024-09-22T03:23:19.184870+0000","last_fullsized":"2024-09-22T03:23:19.184870+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1907,"log_dups_size":0,"ondisk_log_size":1907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1908,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":159,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.191329+0000","last_change":"2024-09-22T03:23:11.117757+0000","last_active":"2024-09-22T03:23:18.191329+0000","last_peered":"2024-09-22T03:23:18.191329+0000","last_clean":"2024-09-22T03:23:18.191329+0000","last_became_active":"2024-09-22T03:23:10.226835+0000","last_became_peered":"2024-09-22T03:23:10.226835+0000","last_unstale":"2024-09-22T03:23:18.191329+0000","last_undegraded":"2024-09-22T03:23:18.191329+0000","last_fullsized":"2024-09-22T03:23:18.191329+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"149'1879","reported_seq":2616,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.045198+0000","last_change":"2024-09-22T03:22:43.211209+0000","last_active":"2024-09-22T03:23:18.045198+0000","last_peered":"2024-09-22T03:23:18.045198+0000","last_clean":"2024-09-22T03:23:18.045198+0000","last_became_active":"2024-09-22T03:22:43.201439+0000","last_became_peered":"2024-09-22T03:22:43.201439+0000","last_unstale":"2024-09-22T03:23:18.045198+0000","last_undegraded":"2024-09-22T03:23:18.045198+0000","last_fullsized":"2024-09-22T03:23:18.045198+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1879,"log_dups_size":0,"ondisk_log_size":1879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1880,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":402,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.173357+0000","last_change":"2024-09-22T03:22:24.447081+0000","last_active":"2024-09-22T03:23:17.173357+0000","last_peered":"2024-09-22T03:23:17.173357+0000","last_clean":"2024-09-22T03:23:17.173357+0000","last_became_active":"2024-09-22T03:22:23.585123+0000","last_became_peered":"2024-09-22T03:22:23.585123+0000","last_unstale":"2024-09-22T03:23:17.173357+0000","last_undegraded":"2024-09-22T03:23:17.173357+0000","last_fullsized":"2024-09-22T03:23:17.173357+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"150'1668","reported_seq":1837,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-22T03:23:19.184027+0000","last_change":"2024-09-22T03:23:10.274356+0000","last_active":"2024-09-22T03:23:19.184027+0000","last_peered":"2024-09-22T03:23:19.184027+0000","last_clean":"2024-09-22T03:23:19.184027+0000","last_became_active":"2024-09-22T03:23:10.215382+0000","last_became_peered":"2024-09-22T03:23:10.215382+0000","last_unstale":"2024-09-22T03:23:19.184027+0000","last_undegraded":"2024-09-22T03:23:19.184027+0000","last_fullsized":"2024-09-22T03:23:19.184027+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1668,"log_dups_size":0,"ondisk_log_size":1668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":295,"num_object_clones":0,"num_object_copies":885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":277,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.195393+0000","last_change":"2024-09-22T03:23:11.109171+0000","last_active":"2024-09-22T03:23:18.195393+0000","last_peered":"2024-09-22T03:23:18.195393+0000","last_clean":"2024-09-22T03:23:18.195393+0000","last_became_active":"2024-09-22T03:23:10.194313+0000","last_became_peered":"2024-09-22T03:23:10.194313+0000","last_unstale":"2024-09-22T03:23:18.195393+0000","last_undegraded":"2024-09-22T03:23:18.195393+0000","last_fullsized":"2024-09-22T03:23:18.195393+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"149'1809","reported_seq":2420,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-22T03:23:17.450391+0000","last_change":"2024-09-22T03:23:10.772811+0000","last_active":"2024-09-22T03:23:17.450391+0000","last_peered":"2024-09-22T03:23:17.450391+0000","last_clean":"2024-09-22T03:23:17.450391+0000","last_became_active":"2024-09-22T03:23:10.188905+0000","last_became_peered":"2024-09-22T03:23:10.188905+0000","last_unstale":"2024-09-22T03:23:17.450391+0000","last_undegraded":"2024-09-22T03:23:17.450391+0000","last_fullsized":"2024-09-22T03:23:17.450391+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1809,"log_dups_size":0,"ondisk_log_size":1809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":320,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":232,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-22T03:23:18.196846+0000","last_change":"2024-09-22T03:23:11.109551+0000","last_active":"2024-09-22T03:23:18.196846+0000","last_peered":"2024-09-22T03:23:18.196846+0000","last_clean":"2024-09-22T03:23:18.196846+0000","last_became_active":"2024-09-22T03:23:10.218679+0000","last_became_peered":"2024-09-22T03:23:10.218679+0000","last_unstale":"2024-09-22T03:23:18.196846+0000","last_undegraded":"2024-09-22T03:23:18.196846+0000","last_fullsized":"2024-09-22T03:23:18.196846+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":315621400,"num_objects":4817,"num_object_clones":0,"num_object_copies":14451,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":29057,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5107,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62111744,"data_stored":993591368,"data_compressed":47181032,"data_compressed_allocated":62099456,"data_compressed_original":993591296,"omap_allocated":0,"internal_metadata":0},"log_size":29044,"ondisk_log_size":29044,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676598967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66908,"kb_used_data":16652,"kb_used_omap":0,"kb_used_meta":50240,"kb_avail":94304932,"statfs":{"total":96636764160,"available":96568250368,"internally_reserved":0,"allocated":17051648,"data_stored":253476743,"data_compressed":12012320,"data_compressed_allocated":15810560,"data_compressed_original":252968960,"omap_allocated":0,"internal_metadata":51445760},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":119,"seq":511101108279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63548,"kb_used_data":19116,"kb_used_omap":0,"kb_used_meta":44416,"kb_avail":94308292,"statfs":{"total":96636764160,"available":96571691008,"internally_reserved":0,"allocated":19574784,"data_stored":293454247,"data_compressed":13912604,"data_compressed_allocated":18313216,"data_compressed_original":292929536,"omap_allocated":0,"internal_metadata":45481984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":144,"seq":618475290633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52548,"kb_used_data":13552,"kb_used_omap":0,"kb_used_meta":38976,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":13877248,"data_stored":202359183,"data_compressed":9586924,"data_compressed_allocated":12619776,"data_compressed_original":201834496,"omap_allocated":0,"internal_metadata":39911424},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79060,"kb_used_data":18692,"kb_used_omap":0,"kb_used_meta":60352,"kb_avail":94292780,"statfs":{"total":96636764160,"available":96555806720,"internally_reserved":0,"allocated":19140608,"data_stored":286445178,"data_compressed":13579620,"data_compressed_allocated":17874944,"data_compressed_original":285917184,"omap_allocated":0,"internal_metadata":61800448},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17154048,"data_stored":274399256,"data_compressed":13029944,"data_compressed_allocated":17149952,"data_compressed_original":274399232,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11898880,"data_stored":190382080,"data_compressed":9040360,"data_compressed_allocated":11898880,"data_compressed_original":190382080,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18141184,"data_stored":290193432,"data_compressed":13779936,"data_compressed_allocated":18137088,"data_compressed_original":290193408,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14917632,"data_stored":238616600,"data_compressed":11330792,"data_compressed_allocated":14913536,"data_compressed_original":238616576,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:21.020 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:23:21.020 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:23:21.020 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 3, 2, 1] 2024-09-22T03:23:21.021 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:23:21.021 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:23:21.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:23:22.162 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:22.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:22.265+0000 7f1d97efa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:22.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:22.269+0000 7f1d97efa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:27.122 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:23:27.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~6s 2024-09-22T03:23:32.123 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 3, 2] 2024-09-22T03:23:32.123 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:23:32.123 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 3, 2] 2024-09-22T03:23:32.124 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:23:32.124 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:23:32.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:23:32.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:23:37.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2024-09-22T03:23:38.225 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:23:43.225 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1, 2] live_osds: [0, 3] 2024-09-22T03:23:43.225 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-22T03:23:43.225 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:23:43.226 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:23:43.228 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:23:43.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:23:43.341 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:23:43.344 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:43.345 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:23:43.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:23:48.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:23:48.442 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:23:48.445 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:48.445 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:23:51.119 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:23:53.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:23:53.547 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:23:53.548 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [] 2024-09-22T03:23:53.548 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:23:53.556 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:23:53.557 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:23:53.557 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:23:53.560 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:23:53.560 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:23:53.670 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:23:53.674 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:53.674 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:23:57.317 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:23:57.317 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:23:58.675 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:23:58.813 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:23:58.816 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:58.817 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:23:59.318 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:59.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:59.467+0000 7fa51f317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:59.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:23:59.475+0000 7fa51f317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:59.542 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:23:59.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:23:59.741 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:24:01.743 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:01.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:01.843+0000 7fa0c1b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:01.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:01.848+0000 7fa0c1b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:03.817 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:24:03.918 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:04.012 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:24:04.012 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "replicated_request", 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16777438, 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "RepRequest": { 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "reqid": "client.4937.0:31101", 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "5.3", 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 161, 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "min_epoch": 159, 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "oid": "5:c9dea711:::benchmark_data_smithi146_42560_object14162:head", 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2" 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.013 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "background_recovery_sub", 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16777437, 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": {} 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "replicated_request", 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554776, 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "RepRequest": { 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "reqid": "client.4937.0:31103", 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "5.2", 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 161, 2024-09-22T03:24:04.014 INFO:teuthology.orchestra.run.smithi146.stdout: "min_epoch": 159, 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "oid": "5:45369665:::benchmark_data_smithi146_42560_object14164:head", 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "0" 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "background_recovery_sub", 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554775, 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": {} 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 101, 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.015 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.578231+0000" 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.578240+0000" 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.016 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.578344+0000" 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.017 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.579216+0000" 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.579521+0000" 2024-09-22T03:24:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.689576+0000" 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.594083+0000" 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.020 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.595206+0000" 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.596268+0000" 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.022 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.023 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.597210+0000" 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.024 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.603654+0000" 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.625746+0000" 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.688916+0000" 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.026 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 109, 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.661165+0000" 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.027 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.661186+0000" 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.661334+0000" 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.028 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.661423+0000" 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.029 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.662038+0000" 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.030 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.728588+0000" 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.662491+0000" 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.662800+0000" 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.663123+0000" 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.033 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.663640+0000" 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.665234+0000" 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.036 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.722827+0000" 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 110, 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.663674+0000" 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.038 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.663685+0000" 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.663856+0000" 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.664086+0000" 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.040 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.665624+0000" 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.041 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.738806+0000" 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.669314+0000" 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.042 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.686089+0000" 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.043 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.686318+0000" 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.044 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.686574+0000" 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.689438+0000" 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.738062+0000" 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.048 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 119, 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.049 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.742537+0000" 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.742543+0000" 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.742657+0000" 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.742713+0000" 2024-09-22T03:24:04.051 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.745566+0000" 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.052 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.807450+0000" 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.745979+0000" 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.053 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.746158+0000" 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.054 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.746577+0000" 2024-09-22T03:24:04.055 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.056 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.057 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.747544+0000" 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.058 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.755930+0000" 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.059 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.806786+0000" 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.060 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.061 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.061 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.061 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.061 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.061 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.061 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.061 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 120, 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.743467+0000" 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.062 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.063 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.743471+0000" 2024-09-22T03:24:04.063 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.063 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.063 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.063 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.063 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.064 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.064 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.064 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.743536+0000" 2024-09-22T03:24:04.064 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.064 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.064 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.064 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.065 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.065 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.065 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.065 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.065 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.065 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.065 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.066 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.066 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.066 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.066 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.066 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.743594+0000" 2024-09-22T03:24:04.066 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.745613+0000" 2024-09-22T03:24:04.068 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.068 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.068 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.068 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.068 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.069 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.069 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.069 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.069 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.069 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.069 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.069 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.070 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.070 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.070 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.070 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.817940+0000" 2024-09-22T03:24:04.070 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.070 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.070 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.071 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.746871+0000" 2024-09-22T03:24:04.071 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.071 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.071 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.071 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.071 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.071 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.072 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.072 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.072 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.072 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.072 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.072 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.073 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.073 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.073 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.073 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.747269+0000" 2024-09-22T03:24:04.073 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.073 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.074 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.074 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.074 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.074 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.074 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.074 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.074 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.075 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.075 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.075 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.075 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.075 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.075 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.076 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.747700+0000" 2024-09-22T03:24:04.076 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.076 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.076 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.076 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.076 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.077 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.077 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.077 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.077 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.077 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.077 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.078 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.078 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.078 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.078 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.078 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.078 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.078 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.079 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.079 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.079 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.079 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.079 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.079 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.079 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.080 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.080 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.080 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.080 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.080 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.080 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.080 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.748286+0000" 2024-09-22T03:24:04.081 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.081 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.081 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.081 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.081 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.081 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.082 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.082 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.082 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.082 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.082 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.082 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.755379+0000" 2024-09-22T03:24:04.082 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.083 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.083 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.083 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.756191+0000" 2024-09-22T03:24:04.083 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.083 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.083 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.084 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.084 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.084 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.084 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.084 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.084 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.084 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.085 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.085 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.085 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.085 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.085 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.085 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.817553+0000" 2024-09-22T03:24:04.086 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.086 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.086 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.086 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.086 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.086 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.086 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.087 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.087 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.087 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.087 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.087 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.087 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.088 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 121, 2024-09-22T03:24:04.088 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.088 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.088 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.088 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.750798+0000" 2024-09-22T03:24:04.088 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.089 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.089 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.089 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.750806+0000" 2024-09-22T03:24:04.089 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.089 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.089 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.089 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.090 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.090 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.090 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.090 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.751698+0000" 2024-09-22T03:24:04.090 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.091 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.091 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.091 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.091 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.091 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.091 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.091 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.092 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.092 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.092 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.092 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.092 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.092 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.093 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.093 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.751799+0000" 2024-09-22T03:24:04.093 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.093 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.093 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.093 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.093 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.094 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.094 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.094 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.753920+0000" 2024-09-22T03:24:04.094 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.094 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.094 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.094 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.095 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.819525+0000" 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.754588+0000" 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.096 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.097 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.755019+0000" 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.098 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.099 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.756100+0000" 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.100 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.101 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.763722+0000" 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.102 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.766007+0000" 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.772109+0000" 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.103 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.819203+0000" 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.104 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 122, 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.754605+0000" 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.105 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.754611+0000" 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.755036+0000" 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.106 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.755103+0000" 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.107 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.756370+0000" 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.108 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.820320+0000" 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.758456+0000" 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.109 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.763861+0000" 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.110 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.764171+0000" 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.111 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.112 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.765239+0000" 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.113 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.766875+0000" 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.114 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.783503+0000" 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.115 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.819887+0000" 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.116 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 132, 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.847707+0000" 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.847720+0000" 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.117 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.849123+0000" 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.118 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.849456+0000" 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.119 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.854153+0000" 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.120 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.896520+0000" 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.854438+0000" 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.121 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.857418+0000" 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.122 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.857798+0000" 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.123 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.124 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.860927+0000" 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.125 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.126 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.862381+0000" 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.127 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:02.895990+0000" 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.128 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 144, 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.008906+0000" 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.008919+0000" 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.129 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.009877+0000" 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.130 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.009987+0000" 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.010164+0000" 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.131 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.043459+0000" 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.132 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.010508+0000" 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.133 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.010686+0000" 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.134 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.010849+0000" 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.135 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.136 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.011117+0000" 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.137 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.012660+0000" 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.138 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.043186+0000" 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.139 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 148, 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.044753+0000" 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.044760+0000" 2024-09-22T03:24:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.045558+0000" 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.141 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.045736+0000" 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.142 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.049931+0000" 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.143 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.089577+0000" 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.050298+0000" 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.144 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.050531+0000" 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.145 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.050739+0000" 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.146 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.147 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.051024+0000" 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.148 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.054027+0000" 2024-09-22T03:24:04.149 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.150 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.089099+0000" 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 150, 2024-09-22T03:24:04.151 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.053663+0000" 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.053674+0000" 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.152 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.053770+0000" 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.153 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.054446+0000" 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.059008+0000" 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.154 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.099307+0000" 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.155 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.059308+0000" 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.156 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.059624+0000" 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.157 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.059736+0000" 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.158 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.159 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.059927+0000" 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.160 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.061670+0000" 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.161 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.098987+0000" 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.162 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 158, 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.122028+0000" 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.163 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.122043+0000" 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.122188+0000" 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.164 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.122256+0000" 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.165 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.123558+0000" 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.166 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.168622+0000" 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.123936+0000" 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.167 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.124120+0000" 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.124286+0000" 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.169 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.170 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.124518+0000" 2024-09-22T03:24:04.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.172 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.173 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.125437+0000" 2024-09-22T03:24:04.173 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.173 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.173 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.173 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.173 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.173 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.174 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.174 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.174 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.174 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.174 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.174 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.175 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.175 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.175 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.175 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.163341+0000" 2024-09-22T03:24:04.175 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.175 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.175 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.176 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.176 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.176 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.176 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.176 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.176 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.176 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.177 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.177 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.177 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.177 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 175, 2024-09-22T03:24:04.177 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.177 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.177 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.178 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.264124+0000" 2024-09-22T03:24:04.178 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.178 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.178 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.178 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.264138+0000" 2024-09-22T03:24:04.178 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.179 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.179 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.179 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.179 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.179 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.179 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.179 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.264572+0000" 2024-09-22T03:24:04.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.180 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.180 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.180 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.180 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.180 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.181 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.181 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.181 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.181 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.181 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.181 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.181 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.182 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.182 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.264624+0000" 2024-09-22T03:24:04.182 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.182 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.182 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.182 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.182 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.183 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.183 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.183 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.264744+0000" 2024-09-22T03:24:04.183 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.183 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.183 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.183 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.184 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.184 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.184 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.184 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.184 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.184 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.185 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.185 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.185 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.185 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.185 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.306322+0000" 2024-09-22T03:24:04.185 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.186 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.186 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.186 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.265093+0000" 2024-09-22T03:24:04.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.186 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.186 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.186 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.187 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.187 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.187 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.187 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.187 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.187 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.188 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.188 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.188 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.188 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.188 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.265399+0000" 2024-09-22T03:24:04.189 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.189 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.189 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.189 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.189 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.189 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.189 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.190 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.190 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.190 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.190 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.190 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.190 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.191 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.191 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.191 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.265578+0000" 2024-09-22T03:24:04.191 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.191 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.191 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.191 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.192 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.192 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.192 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.192 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.192 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.192 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.192 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.193 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.193 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.193 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.193 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.193 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.193 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.193 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.194 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.194 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.194 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.194 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.194 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.194 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.194 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.195 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.195 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.195 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.195 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.195 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.195 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.196 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.265732+0000" 2024-09-22T03:24:04.196 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.196 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.196 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.196 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.196 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.197 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.197 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.197 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.197 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.197 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.197 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.197 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.198 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.198 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.198 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.266659+0000" 2024-09-22T03:24:04.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.198 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.199 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.199 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.199 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.199 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.199 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.199 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.200 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.200 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.200 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.200 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.200 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.201 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.305837+0000" 2024-09-22T03:24:04.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.201 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.201 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.201 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.202 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.202 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.202 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.202 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.202 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.203 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.203 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.203 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 187, 2024-09-22T03:24:04.203 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.203 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.203 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.204 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.339831+0000" 2024-09-22T03:24:04.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.204 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.204 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.204 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.339838+0000" 2024-09-22T03:24:04.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.205 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.205 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.205 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.205 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.205 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.205 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.340104+0000" 2024-09-22T03:24:04.206 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.206 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.206 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.206 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.206 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.207 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.207 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.207 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.207 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.207 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.207 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.207 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.340238+0000" 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.208 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.341461+0000" 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.209 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.373972+0000" 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.341707+0000" 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.210 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.342031+0000" 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.211 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.212 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.342197+0000" 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.214 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.342351+0000" 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.215 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.216 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.343514+0000" 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.217 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.373568+0000" 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.218 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 197, 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.556342+0000" 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.556355+0000" 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.219 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.556478+0000" 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.220 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.556547+0000" 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.556729+0000" 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.222 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.643178+0000" 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.223 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.557165+0000" 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.224 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.557343+0000" 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.225 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.557496+0000" 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.226 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.227 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.557734+0000" 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.228 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.563563+0000" 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.642658+0000" 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.230 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 199, 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.231 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.578158+0000" 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.578165+0000" 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.578259+0000" 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.232 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.578309+0000" 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.233 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.582463+0000" 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.664744+0000" 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.583104+0000" 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.235 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.583364+0000" 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.236 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.583550+0000" 2024-09-22T03:24:04.237 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.238 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.239 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.583935+0000" 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.240 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.586070+0000" 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.241 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.664084+0000" 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.242 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 209, 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.724553+0000" 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.243 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.724560+0000" 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.724647+0000" 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.244 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.724701+0000" 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.724833+0000" 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.246 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.762837+0000" 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.725179+0000" 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.247 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.725455+0000" 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.248 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.725591+0000" 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.725733+0000" 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.251 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.726762+0000" 2024-09-22T03:24:04.252 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.253 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.762344+0000" 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.254 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 211, 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.738894+0000" 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.738901+0000" 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.255 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.738987+0000" 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.739060+0000" 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.742620+0000" 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.772248+0000" 2024-09-22T03:24:04.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.742983+0000" 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.743156+0000" 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.743312+0000" 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.262 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.743563+0000" 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.744692+0000" 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.771821+0000" 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 215, 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.803048+0000" 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:04.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.803060+0000" 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.803188+0000" 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.803278+0000" 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:04.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.803485+0000" 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.848300+0000" 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.803902+0000" 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:04.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.804071+0000" 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:04.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.804222+0000" 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:04.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:04.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.804457+0000" 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.810492+0000" 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:24:03.847860+0000" 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:24:04.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:24:04.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:04.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:04.277 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.277 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.277 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:24:04.277 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:24:04.277 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:24:04.278 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:24:04.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:04.065+0000 7f78e2157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:04.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:04.073+0000 7f78e2157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:04.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:04.430+0000 7f33bc917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:04.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:04.445+0000 7f33bc917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:05.102 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:24:05.123 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:24:05.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:05.252+0000 7f16280fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:05.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:05.269+0000 7f16280fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:06.139 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:24:06.159 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:24:06.305 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:06.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:06.303+0000 7fcc552ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:06.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:06.322+0000 7fcc552ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:06.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:06.472+0000 7f9f75a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:06.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:06.480+0000 7f9f75a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:07.140 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:24:07.159 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:24:07.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:07.285+0000 7f0c45757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:07.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:07.298+0000 7f0c45757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.144 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:24:08.164 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:24:08.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:08.295+0000 7f5c1a263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:08.305+0000 7f5c1a263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.579 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:08.579 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:08.598 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":550,"stamp":"2024-09-22T03:24:08.144700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179503691,"num_objects":2735,"num_object_clones":0,"num_object_copies":8205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":31282,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5659,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31231,"ondisk_log_size":31231,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":235936,"kb_used_data":40412,"kb_used_omap":0,"kb_used_meta":195456,"kb_avail":377251424,"statfs":{"total":386547056640,"available":386305458176,"internally_reserved":0,"allocated":41381888,"data_stored":577539677,"data_compressed":27326140,"data_compressed_allocated":35971072,"data_compressed_original":575291392,"omap_allocated":0,"internal_metadata":200146944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55377920,"num_objects":-845,"num_object_clones":0,"num_object_copies":-2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":845,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.754519"},"pg_stats":[{"pgid":"5.f","version":"161'1889","reported_seq":2049,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.988530+0000","last_change":"2024-09-22T03:23:56.200281+0000","last_active":"2024-09-22T03:24:03.988530+0000","last_peered":"2024-09-22T03:24:03.988530+0000","last_clean":"2024-09-22T03:24:03.988530+0000","last_became_active":"2024-09-22T03:23:56.190271+0000","last_became_peered":"2024-09-22T03:23:56.190271+0000","last_unstale":"2024-09-22T03:24:03.988530+0000","last_undegraded":"2024-09-22T03:24:03.988530+0000","last_fullsized":"2024-09-22T03:24:03.988530+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1889,"log_dups_size":0,"ondisk_log_size":1889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10223616,"num_objects":156,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1890,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"161'1925","reported_seq":2722,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:04.215587+0000","last_change":"2024-09-22T03:23:56.259934+0000","last_active":"2024-09-22T03:24:04.215587+0000","last_peered":"2024-09-22T03:24:04.215587+0000","last_clean":"2024-09-22T03:24:04.215587+0000","last_became_active":"2024-09-22T03:23:56.182078+0000","last_became_peered":"2024-09-22T03:23:56.182078+0000","last_unstale":"2024-09-22T03:24:04.215587+0000","last_undegraded":"2024-09-22T03:24:04.215587+0000","last_fullsized":"2024-09-22T03:24:04.215587+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1925,"log_dups_size":0,"ondisk_log_size":1925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1926,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"161'1956","reported_seq":2730,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:04.211429+0000","last_change":"2024-09-22T03:23:56.218248+0000","last_active":"2024-09-22T03:24:04.211429+0000","last_peered":"2024-09-22T03:24:04.211429+0000","last_clean":"2024-09-22T03:24:04.211429+0000","last_became_active":"2024-09-22T03:23:56.211834+0000","last_became_peered":"2024-09-22T03:23:56.211834+0000","last_unstale":"2024-09-22T03:24:04.211429+0000","last_undegraded":"2024-09-22T03:24:04.211429+0000","last_fullsized":"2024-09-22T03:24:04.211429+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1956,"log_dups_size":0,"ondisk_log_size":1956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010072,"num_objects":169,"num_object_clones":0,"num_object_copies":507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1957,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":196632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"160'1977","reported_seq":2856,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.137020+0000","last_change":"2024-09-22T03:24:03.096670+0000","last_active":"2024-09-22T03:24:03.137020+0000","last_peered":"2024-09-22T03:24:03.137020+0000","last_clean":"2024-09-22T03:24:03.137020+0000","last_became_active":"2024-09-22T03:24:02.210115+0000","last_became_peered":"2024-09-22T03:24:02.210115+0000","last_unstale":"2024-09-22T03:24:03.137020+0000","last_undegraded":"2024-09-22T03:24:03.137020+0000","last_fullsized":"2024-09-22T03:24:03.137020+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1977,"log_dups_size":0,"ondisk_log_size":1977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":169,"num_object_clones":0,"num_object_copies":507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"161'1942","reported_seq":2242,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.186155+0000","last_change":"2024-09-22T03:24:02.498490+0000","last_active":"2024-09-22T03:24:04.186155+0000","last_peered":"2024-09-22T03:24:04.186155+0000","last_clean":"2024-09-22T03:23:21.078366+0000","last_became_active":"2024-09-22T03:24:02.350737+0000","last_became_peered":"2024-09-22T03:24:02.350737+0000","last_unstale":"2024-09-22T03:24:04.186155+0000","last_undegraded":"2024-09-22T03:24:04.186155+0000","last_fullsized":"2024-09-22T03:24:04.186155+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1942,"log_dups_size":0,"ondisk_log_size":1942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10551296,"num_objects":161,"num_object_clones":0,"num_object_copies":483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1943,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,1,3","objects":161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"161'1943","reported_seq":2100,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:04.048737+0000","last_change":"2024-09-22T03:23:56.225812+0000","last_active":"2024-09-22T03:24:04.048737+0000","last_peered":"2024-09-22T03:24:04.048737+0000","last_clean":"2024-09-22T03:24:04.048737+0000","last_became_active":"2024-09-22T03:23:56.219474+0000","last_became_peered":"2024-09-22T03:23:56.219474+0000","last_unstale":"2024-09-22T03:24:04.048737+0000","last_undegraded":"2024-09-22T03:24:04.048737+0000","last_fullsized":"2024-09-22T03:24:04.048737+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1943,"log_dups_size":0,"ondisk_log_size":1943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":176,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1944,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"163'2022","reported_seq":2798,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:07.025354+0000","last_change":"2024-09-22T03:24:03.164828+0000","last_active":"2024-09-22T03:24:07.025354+0000","last_peered":"2024-09-22T03:24:07.025354+0000","last_clean":"2024-09-22T03:24:07.025354+0000","last_became_active":"2024-09-22T03:24:02.321096+0000","last_became_peered":"2024-09-22T03:24:02.321096+0000","last_unstale":"2024-09-22T03:24:07.025354+0000","last_undegraded":"2024-09-22T03:24:07.025354+0000","last_fullsized":"2024-09-22T03:24:07.025354+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2022,"log_dups_size":0,"ondisk_log_size":2022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2023,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"158'1923","reported_seq":5360,"reported_epoch":161,"state":"activating","last_fresh":"2024-09-22T03:24:03.138977+0000","last_change":"2024-09-22T03:24:03.168146+0000","last_active":"2024-09-22T03:24:01.151286+0000","last_peered":"2024-09-22T03:24:01.121673+0000","last_clean":"2024-09-22T03:23:21.158231+0000","last_became_active":"2024-09-22T03:23:57.229727+0000","last_became_peered":"2024-09-22T03:23:57.229727+0000","last_unstale":"2024-09-22T03:24:03.138977+0000","last_undegraded":"2024-09-22T03:24:03.138977+0000","last_fullsized":"2024-09-22T03:24:03.138977+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1923,"log_dups_size":0,"ondisk_log_size":1923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":201,"num_object_clones":0,"num_object_copies":603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1923,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":19726336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":["3","0","2"],"object_location_counts":[{"shards":"1,2,3","objects":201}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"161'1875","reported_seq":2625,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.120681+0000","last_change":"2024-09-22T03:24:03.721722+0000","last_active":"2024-09-22T03:24:04.120681+0000","last_peered":"2024-09-22T03:24:04.120681+0000","last_clean":"2024-09-22T03:23:21.041111+0000","last_became_active":"2024-09-22T03:24:02.313406+0000","last_became_peered":"2024-09-22T03:24:02.313406+0000","last_unstale":"2024-09-22T03:24:04.120681+0000","last_undegraded":"2024-09-22T03:24:04.120681+0000","last_fullsized":"2024-09-22T03:24:04.120681+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1875,"log_dups_size":0,"ondisk_log_size":1875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10747904,"num_objects":164,"num_object_clones":0,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1876,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,1,2","objects":164}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":259,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.140859+0000","last_change":"2024-09-22T03:23:57.118798+0000","last_active":"2024-09-22T03:24:03.140859+0000","last_peered":"2024-09-22T03:24:03.140859+0000","last_clean":"2024-09-22T03:24:03.140859+0000","last_became_active":"2024-09-22T03:23:56.167489+0000","last_became_peered":"2024-09-22T03:23:56.167489+0000","last_unstale":"2024-09-22T03:24:03.140859+0000","last_undegraded":"2024-09-22T03:24:03.140859+0000","last_fullsized":"2024-09-22T03:24:03.140859+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"161'1978","reported_seq":2732,"reported_epoch":161,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:24:04.047463+0000","last_change":"2024-09-22T03:24:02.389026+0000","last_active":"2024-09-22T03:24:04.047463+0000","last_peered":"2024-09-22T03:24:04.047463+0000","last_clean":"2024-09-22T03:23:21.123066+0000","last_became_active":"2024-09-22T03:24:02.263994+0000","last_became_peered":"2024-09-22T03:24:02.263994+0000","last_unstale":"2024-09-22T03:24:04.047463+0000","last_undegraded":"2024-09-22T03:24:02.387785+0000","last_fullsized":"2024-09-22T03:24:04.047463+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1978,"log_dups_size":0,"ondisk_log_size":1978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11599872,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1979,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"1,2,3","objects":177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":379,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.143894+0000","last_change":"2024-09-22T03:24:03.145088+0000","last_active":"2024-09-22T03:24:03.143894+0000","last_peered":"2024-09-22T03:24:03.143894+0000","last_clean":"2024-09-22T03:24:03.143894+0000","last_became_active":"2024-09-22T03:24:02.330151+0000","last_became_peered":"2024-09-22T03:24:02.330151+0000","last_unstale":"2024-09-22T03:24:03.143894+0000","last_undegraded":"2024-09-22T03:24:03.143894+0000","last_fullsized":"2024-09-22T03:24:03.143894+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"161'2016","reported_seq":2589,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.953238+0000","last_change":"2024-09-22T03:23:56.211523+0000","last_active":"2024-09-22T03:24:03.953238+0000","last_peered":"2024-09-22T03:24:03.953238+0000","last_clean":"2024-09-22T03:24:03.953238+0000","last_became_active":"2024-09-22T03:23:56.208082+0000","last_became_peered":"2024-09-22T03:23:56.208082+0000","last_unstale":"2024-09-22T03:24:03.953238+0000","last_undegraded":"2024-09-22T03:24:03.953238+0000","last_fullsized":"2024-09-22T03:24:03.953238+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2016,"log_dups_size":0,"ondisk_log_size":2016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11337728,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2017,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":427,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.151836+0000","last_change":"2024-09-22T03:24:03.155372+0000","last_active":"2024-09-22T03:24:03.151836+0000","last_peered":"2024-09-22T03:24:03.151836+0000","last_clean":"2024-09-22T03:24:03.151836+0000","last_became_active":"2024-09-22T03:24:02.225075+0000","last_became_peered":"2024-09-22T03:24:02.225075+0000","last_unstale":"2024-09-22T03:24:03.151836+0000","last_undegraded":"2024-09-22T03:24:03.151836+0000","last_fullsized":"2024-09-22T03:24:03.151836+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"161'1915","reported_seq":2664,"reported_epoch":161,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:24:04.090857+0000","last_change":"2024-09-22T03:24:02.372370+0000","last_active":"2024-09-22T03:24:04.090857+0000","last_peered":"2024-09-22T03:24:04.090857+0000","last_clean":"2024-09-22T03:23:20.988947+0000","last_became_active":"2024-09-22T03:24:02.365735+0000","last_became_peered":"2024-09-22T03:24:02.365735+0000","last_unstale":"2024-09-22T03:24:04.090857+0000","last_undegraded":"2024-09-22T03:24:02.267580+0000","last_fullsized":"2024-09-22T03:24:04.090857+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1915,"log_dups_size":0,"ondisk_log_size":1915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10223616,"num_objects":156,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1916,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,1,2","objects":156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":340,"reported_epoch":161,"state":"active+clean+snaptrim","last_fresh":"2024-09-22T03:24:03.140242+0000","last_change":"2024-09-22T03:24:03.140242+0000","last_active":"2024-09-22T03:24:03.140242+0000","last_peered":"2024-09-22T03:24:03.140242+0000","last_clean":"2024-09-22T03:24:03.140242+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:03.140242+0000","last_undegraded":"2024-09-22T03:24:03.140242+0000","last_fullsized":"2024-09-22T03:24:03.140242+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":498,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.141794+0000","last_change":"2024-09-22T03:24:02.377618+0000","last_active":"2024-09-22T03:24:03.141794+0000","last_peered":"2024-09-22T03:24:03.141794+0000","last_clean":"2024-09-22T03:24:03.141794+0000","last_became_active":"2024-09-22T03:24:02.362178+0000","last_became_peered":"2024-09-22T03:24:02.362178+0000","last_unstale":"2024-09-22T03:24:03.141794+0000","last_undegraded":"2024-09-22T03:24:03.141794+0000","last_fullsized":"2024-09-22T03:24:03.141794+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"161'2025","reported_seq":2893,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.171221+0000","last_change":"2024-09-22T03:24:02.489697+0000","last_active":"2024-09-22T03:24:04.171221+0000","last_peered":"2024-09-22T03:24:04.171221+0000","last_clean":"2024-09-22T03:23:21.033659+0000","last_became_active":"2024-09-22T03:24:02.328698+0000","last_became_peered":"2024-09-22T03:24:02.328698+0000","last_unstale":"2024-09-22T03:24:04.171221+0000","last_undegraded":"2024-09-22T03:24:04.171221+0000","last_fullsized":"2024-09-22T03:24:04.171221+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2025,"log_dups_size":0,"ondisk_log_size":2025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058624,"num_objects":184,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2026,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,1,2","objects":184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":164,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.169093+0000","last_change":"2024-09-22T03:24:03.169310+0000","last_active":"2024-09-22T03:24:06.169093+0000","last_peered":"2024-09-22T03:24:06.169093+0000","last_clean":"2024-09-22T03:24:06.169093+0000","last_became_active":"2024-09-22T03:24:02.324336+0000","last_became_peered":"2024-09-22T03:24:02.324336+0000","last_unstale":"2024-09-22T03:24:06.169093+0000","last_undegraded":"2024-09-22T03:24:06.169093+0000","last_fullsized":"2024-09-22T03:24:06.169093+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"160'2016","reported_seq":2789,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.139767+0000","last_change":"2024-09-22T03:23:56.170283+0000","last_active":"2024-09-22T03:24:03.139767+0000","last_peered":"2024-09-22T03:24:03.139767+0000","last_clean":"2024-09-22T03:24:03.139767+0000","last_became_active":"2024-09-22T03:23:56.158439+0000","last_became_peered":"2024-09-22T03:23:56.158439+0000","last_unstale":"2024-09-22T03:24:03.139767+0000","last_undegraded":"2024-09-22T03:24:03.139767+0000","last_fullsized":"2024-09-22T03:24:03.139767+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2016,"log_dups_size":0,"ondisk_log_size":2016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12189696,"num_objects":186,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":336,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.148424+0000","last_change":"2024-09-22T03:23:57.117265+0000","last_active":"2024-09-22T03:24:03.148424+0000","last_peered":"2024-09-22T03:24:03.148424+0000","last_clean":"2024-09-22T03:24:03.148424+0000","last_became_active":"2024-09-22T03:23:56.204767+0000","last_became_peered":"2024-09-22T03:23:56.204767+0000","last_unstale":"2024-09-22T03:24:03.148424+0000","last_undegraded":"2024-09-22T03:24:03.148424+0000","last_fullsized":"2024-09-22T03:24:03.148424+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"161'1782","reported_seq":2069,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.987473+0000","last_change":"2024-09-22T03:24:03.658213+0000","last_active":"2024-09-22T03:24:03.987473+0000","last_peered":"2024-09-22T03:24:03.987473+0000","last_clean":"2024-09-22T03:24:03.987473+0000","last_became_active":"2024-09-22T03:24:02.289524+0000","last_became_peered":"2024-09-22T03:24:02.289524+0000","last_unstale":"2024-09-22T03:24:03.987473+0000","last_undegraded":"2024-09-22T03:24:03.987473+0000","last_fullsized":"2024-09-22T03:24:03.987473+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1782,"log_dups_size":0,"ondisk_log_size":1782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11862016,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":282,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.166385+0000","last_change":"2024-09-22T03:24:03.155475+0000","last_active":"2024-09-22T03:24:06.166385+0000","last_peered":"2024-09-22T03:24:06.166385+0000","last_clean":"2024-09-22T03:24:06.166385+0000","last_became_active":"2024-09-22T03:24:02.266145+0000","last_became_peered":"2024-09-22T03:24:02.266145+0000","last_unstale":"2024-09-22T03:24:06.166385+0000","last_undegraded":"2024-09-22T03:24:06.166385+0000","last_fullsized":"2024-09-22T03:24:06.166385+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"161'1947","reported_seq":2657,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.108924+0000","last_change":"2024-09-22T03:24:03.148285+0000","last_active":"2024-09-22T03:24:04.108924+0000","last_peered":"2024-09-22T03:24:04.108924+0000","last_clean":"2024-09-22T03:23:20.967292+0000","last_became_active":"2024-09-22T03:24:02.316697+0000","last_became_peered":"2024-09-22T03:24:02.316697+0000","last_unstale":"2024-09-22T03:24:04.108924+0000","last_undegraded":"2024-09-22T03:24:04.108924+0000","last_fullsized":"2024-09-22T03:24:04.108924+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1947,"log_dups_size":0,"ondisk_log_size":1947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11993088,"num_objects":183,"num_object_clones":0,"num_object_copies":549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1947,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"1,2,3","objects":183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":237,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.167664+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:06.167664+0000","last_peered":"2024-09-22T03:24:06.167664+0000","last_clean":"2024-09-22T03:24:06.167664+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:06.167664+0000","last_undegraded":"2024-09-22T03:24:06.167664+0000","last_fullsized":"2024-09-22T03:24:06.167664+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":178913304,"num_objects":2731,"num_object_clones":0,"num_object_copies":8193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":31143,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5659,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33992704,"data_stored":543686728,"data_compressed":25817152,"data_compressed_allocated":33980416,"data_compressed_original":543686656,"omap_allocated":0,"internal_metadata":0},"log_size":31131,"ondisk_log_size":31131,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65128,"kb_used_data":10712,"kb_used_omap":0,"kb_used_meta":54400,"kb_avail":94306712,"statfs":{"total":96636764160,"available":96570073088,"internally_reserved":0,"allocated":10969088,"data_stored":154752292,"data_compressed":7322536,"data_compressed_allocated":9637888,"data_compressed_original":154206208,"omap_allocated":0,"internal_metadata":55705600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":156,"seq":670014898185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52432,"kb_used_data":10944,"kb_used_omap":0,"kb_used_meta":41472,"kb_avail":94319408,"statfs":{"total":96636764160,"available":96583073792,"internally_reserved":0,"allocated":11206656,"data_stored":158032938,"data_compressed":7480100,"data_compressed_allocated":9846784,"data_compressed_original":157466624,"omap_allocated":0,"internal_metadata":42467328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54520,"kb_used_data":6948,"kb_used_omap":0,"kb_used_meta":47552,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":7114752,"data_stored":92500216,"data_compressed":4368100,"data_compressed_allocated":5750784,"data_compressed_original":91930624,"omap_allocated":0,"internal_metadata":48693248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63856,"kb_used_data":11808,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":12091392,"data_stored":172254231,"data_compressed":8155404,"data_compressed_allocated":10735616,"data_compressed_original":171687936,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10153984,"data_stored":162398232,"data_compressed":7711536,"data_compressed_allocated":10149888,"data_compressed_original":162398208,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4866048,"data_stored":77856768,"data_compressed":3697056,"data_compressed_allocated":4866048,"data_compressed_original":77856768,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9789440,"data_stored":156565528,"data_compressed":7434568,"data_compressed_allocated":9785344,"data_compressed_original":156565504,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9183232,"data_stored":146866200,"data_compressed":6973992,"data_compressed_allocated":9179136,"data_compressed_original":146866176,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:08.600 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:08.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:08.736+0000 7f69e9b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:08.745+0000 7f69e9b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.847 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:08.990 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:08.991 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:08.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:08.993+0000 7efc13d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:09.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:09.001+0000 7efc13d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:09.009 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":550,"stamp":"2024-09-22T03:24:08.144700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179503691,"num_objects":2735,"num_object_clones":0,"num_object_copies":8205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":31282,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5659,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31231,"ondisk_log_size":31231,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":235936,"kb_used_data":40412,"kb_used_omap":0,"kb_used_meta":195456,"kb_avail":377251424,"statfs":{"total":386547056640,"available":386305458176,"internally_reserved":0,"allocated":41381888,"data_stored":577539677,"data_compressed":27326140,"data_compressed_allocated":35971072,"data_compressed_original":575291392,"omap_allocated":0,"internal_metadata":200146944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55377920,"num_objects":-845,"num_object_clones":0,"num_object_copies":-2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":845,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.754519"},"pg_stats":[{"pgid":"5.f","version":"161'1889","reported_seq":2049,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.988530+0000","last_change":"2024-09-22T03:23:56.200281+0000","last_active":"2024-09-22T03:24:03.988530+0000","last_peered":"2024-09-22T03:24:03.988530+0000","last_clean":"2024-09-22T03:24:03.988530+0000","last_became_active":"2024-09-22T03:23:56.190271+0000","last_became_peered":"2024-09-22T03:23:56.190271+0000","last_unstale":"2024-09-22T03:24:03.988530+0000","last_undegraded":"2024-09-22T03:24:03.988530+0000","last_fullsized":"2024-09-22T03:24:03.988530+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1889,"log_dups_size":0,"ondisk_log_size":1889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10223616,"num_objects":156,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1890,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"161'1925","reported_seq":2722,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:04.215587+0000","last_change":"2024-09-22T03:23:56.259934+0000","last_active":"2024-09-22T03:24:04.215587+0000","last_peered":"2024-09-22T03:24:04.215587+0000","last_clean":"2024-09-22T03:24:04.215587+0000","last_became_active":"2024-09-22T03:23:56.182078+0000","last_became_peered":"2024-09-22T03:23:56.182078+0000","last_unstale":"2024-09-22T03:24:04.215587+0000","last_undegraded":"2024-09-22T03:24:04.215587+0000","last_fullsized":"2024-09-22T03:24:04.215587+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1925,"log_dups_size":0,"ondisk_log_size":1925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1926,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"161'1956","reported_seq":2730,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:04.211429+0000","last_change":"2024-09-22T03:23:56.218248+0000","last_active":"2024-09-22T03:24:04.211429+0000","last_peered":"2024-09-22T03:24:04.211429+0000","last_clean":"2024-09-22T03:24:04.211429+0000","last_became_active":"2024-09-22T03:23:56.211834+0000","last_became_peered":"2024-09-22T03:23:56.211834+0000","last_unstale":"2024-09-22T03:24:04.211429+0000","last_undegraded":"2024-09-22T03:24:04.211429+0000","last_fullsized":"2024-09-22T03:24:04.211429+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1956,"log_dups_size":0,"ondisk_log_size":1956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010072,"num_objects":169,"num_object_clones":0,"num_object_copies":507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1957,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":196632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"160'1977","reported_seq":2856,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.137020+0000","last_change":"2024-09-22T03:24:03.096670+0000","last_active":"2024-09-22T03:24:03.137020+0000","last_peered":"2024-09-22T03:24:03.137020+0000","last_clean":"2024-09-22T03:24:03.137020+0000","last_became_active":"2024-09-22T03:24:02.210115+0000","last_became_peered":"2024-09-22T03:24:02.210115+0000","last_unstale":"2024-09-22T03:24:03.137020+0000","last_undegraded":"2024-09-22T03:24:03.137020+0000","last_fullsized":"2024-09-22T03:24:03.137020+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1977,"log_dups_size":0,"ondisk_log_size":1977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":169,"num_object_clones":0,"num_object_copies":507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"161'1942","reported_seq":2242,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.186155+0000","last_change":"2024-09-22T03:24:02.498490+0000","last_active":"2024-09-22T03:24:04.186155+0000","last_peered":"2024-09-22T03:24:04.186155+0000","last_clean":"2024-09-22T03:23:21.078366+0000","last_became_active":"2024-09-22T03:24:02.350737+0000","last_became_peered":"2024-09-22T03:24:02.350737+0000","last_unstale":"2024-09-22T03:24:04.186155+0000","last_undegraded":"2024-09-22T03:24:04.186155+0000","last_fullsized":"2024-09-22T03:24:04.186155+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1942,"log_dups_size":0,"ondisk_log_size":1942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10551296,"num_objects":161,"num_object_clones":0,"num_object_copies":483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1943,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,1,3","objects":161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"161'1943","reported_seq":2100,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:04.048737+0000","last_change":"2024-09-22T03:23:56.225812+0000","last_active":"2024-09-22T03:24:04.048737+0000","last_peered":"2024-09-22T03:24:04.048737+0000","last_clean":"2024-09-22T03:24:04.048737+0000","last_became_active":"2024-09-22T03:23:56.219474+0000","last_became_peered":"2024-09-22T03:23:56.219474+0000","last_unstale":"2024-09-22T03:24:04.048737+0000","last_undegraded":"2024-09-22T03:24:04.048737+0000","last_fullsized":"2024-09-22T03:24:04.048737+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1943,"log_dups_size":0,"ondisk_log_size":1943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":176,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1944,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"163'2022","reported_seq":2798,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:07.025354+0000","last_change":"2024-09-22T03:24:03.164828+0000","last_active":"2024-09-22T03:24:07.025354+0000","last_peered":"2024-09-22T03:24:07.025354+0000","last_clean":"2024-09-22T03:24:07.025354+0000","last_became_active":"2024-09-22T03:24:02.321096+0000","last_became_peered":"2024-09-22T03:24:02.321096+0000","last_unstale":"2024-09-22T03:24:07.025354+0000","last_undegraded":"2024-09-22T03:24:07.025354+0000","last_fullsized":"2024-09-22T03:24:07.025354+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2022,"log_dups_size":0,"ondisk_log_size":2022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2023,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"158'1923","reported_seq":5360,"reported_epoch":161,"state":"activating","last_fresh":"2024-09-22T03:24:03.138977+0000","last_change":"2024-09-22T03:24:03.168146+0000","last_active":"2024-09-22T03:24:01.151286+0000","last_peered":"2024-09-22T03:24:01.121673+0000","last_clean":"2024-09-22T03:23:21.158231+0000","last_became_active":"2024-09-22T03:23:57.229727+0000","last_became_peered":"2024-09-22T03:23:57.229727+0000","last_unstale":"2024-09-22T03:24:03.138977+0000","last_undegraded":"2024-09-22T03:24:03.138977+0000","last_fullsized":"2024-09-22T03:24:03.138977+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1923,"log_dups_size":0,"ondisk_log_size":1923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":201,"num_object_clones":0,"num_object_copies":603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1923,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":19726336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":["3","0","2"],"object_location_counts":[{"shards":"1,2,3","objects":201}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"161'1875","reported_seq":2625,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.120681+0000","last_change":"2024-09-22T03:24:03.721722+0000","last_active":"2024-09-22T03:24:04.120681+0000","last_peered":"2024-09-22T03:24:04.120681+0000","last_clean":"2024-09-22T03:23:21.041111+0000","last_became_active":"2024-09-22T03:24:02.313406+0000","last_became_peered":"2024-09-22T03:24:02.313406+0000","last_unstale":"2024-09-22T03:24:04.120681+0000","last_undegraded":"2024-09-22T03:24:04.120681+0000","last_fullsized":"2024-09-22T03:24:04.120681+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1875,"log_dups_size":0,"ondisk_log_size":1875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10747904,"num_objects":164,"num_object_clones":0,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1876,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,1,2","objects":164}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":259,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.140859+0000","last_change":"2024-09-22T03:23:57.118798+0000","last_active":"2024-09-22T03:24:03.140859+0000","last_peered":"2024-09-22T03:24:03.140859+0000","last_clean":"2024-09-22T03:24:03.140859+0000","last_became_active":"2024-09-22T03:23:56.167489+0000","last_became_peered":"2024-09-22T03:23:56.167489+0000","last_unstale":"2024-09-22T03:24:03.140859+0000","last_undegraded":"2024-09-22T03:24:03.140859+0000","last_fullsized":"2024-09-22T03:24:03.140859+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"161'1978","reported_seq":2732,"reported_epoch":161,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:24:04.047463+0000","last_change":"2024-09-22T03:24:02.389026+0000","last_active":"2024-09-22T03:24:04.047463+0000","last_peered":"2024-09-22T03:24:04.047463+0000","last_clean":"2024-09-22T03:23:21.123066+0000","last_became_active":"2024-09-22T03:24:02.263994+0000","last_became_peered":"2024-09-22T03:24:02.263994+0000","last_unstale":"2024-09-22T03:24:04.047463+0000","last_undegraded":"2024-09-22T03:24:02.387785+0000","last_fullsized":"2024-09-22T03:24:04.047463+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1978,"log_dups_size":0,"ondisk_log_size":1978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11599872,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1979,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"1,2,3","objects":177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":379,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.143894+0000","last_change":"2024-09-22T03:24:03.145088+0000","last_active":"2024-09-22T03:24:03.143894+0000","last_peered":"2024-09-22T03:24:03.143894+0000","last_clean":"2024-09-22T03:24:03.143894+0000","last_became_active":"2024-09-22T03:24:02.330151+0000","last_became_peered":"2024-09-22T03:24:02.330151+0000","last_unstale":"2024-09-22T03:24:03.143894+0000","last_undegraded":"2024-09-22T03:24:03.143894+0000","last_fullsized":"2024-09-22T03:24:03.143894+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"161'2016","reported_seq":2589,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.953238+0000","last_change":"2024-09-22T03:23:56.211523+0000","last_active":"2024-09-22T03:24:03.953238+0000","last_peered":"2024-09-22T03:24:03.953238+0000","last_clean":"2024-09-22T03:24:03.953238+0000","last_became_active":"2024-09-22T03:23:56.208082+0000","last_became_peered":"2024-09-22T03:23:56.208082+0000","last_unstale":"2024-09-22T03:24:03.953238+0000","last_undegraded":"2024-09-22T03:24:03.953238+0000","last_fullsized":"2024-09-22T03:24:03.953238+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2016,"log_dups_size":0,"ondisk_log_size":2016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11337728,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2017,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":427,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.151836+0000","last_change":"2024-09-22T03:24:03.155372+0000","last_active":"2024-09-22T03:24:03.151836+0000","last_peered":"2024-09-22T03:24:03.151836+0000","last_clean":"2024-09-22T03:24:03.151836+0000","last_became_active":"2024-09-22T03:24:02.225075+0000","last_became_peered":"2024-09-22T03:24:02.225075+0000","last_unstale":"2024-09-22T03:24:03.151836+0000","last_undegraded":"2024-09-22T03:24:03.151836+0000","last_fullsized":"2024-09-22T03:24:03.151836+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"161'1915","reported_seq":2664,"reported_epoch":161,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:24:04.090857+0000","last_change":"2024-09-22T03:24:02.372370+0000","last_active":"2024-09-22T03:24:04.090857+0000","last_peered":"2024-09-22T03:24:04.090857+0000","last_clean":"2024-09-22T03:23:20.988947+0000","last_became_active":"2024-09-22T03:24:02.365735+0000","last_became_peered":"2024-09-22T03:24:02.365735+0000","last_unstale":"2024-09-22T03:24:04.090857+0000","last_undegraded":"2024-09-22T03:24:02.267580+0000","last_fullsized":"2024-09-22T03:24:04.090857+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1915,"log_dups_size":0,"ondisk_log_size":1915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10223616,"num_objects":156,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1916,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,1,2","objects":156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":340,"reported_epoch":161,"state":"active+clean+snaptrim","last_fresh":"2024-09-22T03:24:03.140242+0000","last_change":"2024-09-22T03:24:03.140242+0000","last_active":"2024-09-22T03:24:03.140242+0000","last_peered":"2024-09-22T03:24:03.140242+0000","last_clean":"2024-09-22T03:24:03.140242+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:03.140242+0000","last_undegraded":"2024-09-22T03:24:03.140242+0000","last_fullsized":"2024-09-22T03:24:03.140242+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":498,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.141794+0000","last_change":"2024-09-22T03:24:02.377618+0000","last_active":"2024-09-22T03:24:03.141794+0000","last_peered":"2024-09-22T03:24:03.141794+0000","last_clean":"2024-09-22T03:24:03.141794+0000","last_became_active":"2024-09-22T03:24:02.362178+0000","last_became_peered":"2024-09-22T03:24:02.362178+0000","last_unstale":"2024-09-22T03:24:03.141794+0000","last_undegraded":"2024-09-22T03:24:03.141794+0000","last_fullsized":"2024-09-22T03:24:03.141794+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"161'2025","reported_seq":2893,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.171221+0000","last_change":"2024-09-22T03:24:02.489697+0000","last_active":"2024-09-22T03:24:04.171221+0000","last_peered":"2024-09-22T03:24:04.171221+0000","last_clean":"2024-09-22T03:23:21.033659+0000","last_became_active":"2024-09-22T03:24:02.328698+0000","last_became_peered":"2024-09-22T03:24:02.328698+0000","last_unstale":"2024-09-22T03:24:04.171221+0000","last_undegraded":"2024-09-22T03:24:04.171221+0000","last_fullsized":"2024-09-22T03:24:04.171221+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2025,"log_dups_size":0,"ondisk_log_size":2025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058624,"num_objects":184,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2026,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,1,2","objects":184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":164,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.169093+0000","last_change":"2024-09-22T03:24:03.169310+0000","last_active":"2024-09-22T03:24:06.169093+0000","last_peered":"2024-09-22T03:24:06.169093+0000","last_clean":"2024-09-22T03:24:06.169093+0000","last_became_active":"2024-09-22T03:24:02.324336+0000","last_became_peered":"2024-09-22T03:24:02.324336+0000","last_unstale":"2024-09-22T03:24:06.169093+0000","last_undegraded":"2024-09-22T03:24:06.169093+0000","last_fullsized":"2024-09-22T03:24:06.169093+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"160'2016","reported_seq":2789,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.139767+0000","last_change":"2024-09-22T03:23:56.170283+0000","last_active":"2024-09-22T03:24:03.139767+0000","last_peered":"2024-09-22T03:24:03.139767+0000","last_clean":"2024-09-22T03:24:03.139767+0000","last_became_active":"2024-09-22T03:23:56.158439+0000","last_became_peered":"2024-09-22T03:23:56.158439+0000","last_unstale":"2024-09-22T03:24:03.139767+0000","last_undegraded":"2024-09-22T03:24:03.139767+0000","last_fullsized":"2024-09-22T03:24:03.139767+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2016,"log_dups_size":0,"ondisk_log_size":2016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12189696,"num_objects":186,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":336,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.148424+0000","last_change":"2024-09-22T03:23:57.117265+0000","last_active":"2024-09-22T03:24:03.148424+0000","last_peered":"2024-09-22T03:24:03.148424+0000","last_clean":"2024-09-22T03:24:03.148424+0000","last_became_active":"2024-09-22T03:23:56.204767+0000","last_became_peered":"2024-09-22T03:23:56.204767+0000","last_unstale":"2024-09-22T03:24:03.148424+0000","last_undegraded":"2024-09-22T03:24:03.148424+0000","last_fullsized":"2024-09-22T03:24:03.148424+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"161'1782","reported_seq":2069,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-22T03:24:03.987473+0000","last_change":"2024-09-22T03:24:03.658213+0000","last_active":"2024-09-22T03:24:03.987473+0000","last_peered":"2024-09-22T03:24:03.987473+0000","last_clean":"2024-09-22T03:24:03.987473+0000","last_became_active":"2024-09-22T03:24:02.289524+0000","last_became_peered":"2024-09-22T03:24:02.289524+0000","last_unstale":"2024-09-22T03:24:03.987473+0000","last_undegraded":"2024-09-22T03:24:03.987473+0000","last_fullsized":"2024-09-22T03:24:03.987473+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1782,"log_dups_size":0,"ondisk_log_size":1782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11862016,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":282,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.166385+0000","last_change":"2024-09-22T03:24:03.155475+0000","last_active":"2024-09-22T03:24:06.166385+0000","last_peered":"2024-09-22T03:24:06.166385+0000","last_clean":"2024-09-22T03:24:06.166385+0000","last_became_active":"2024-09-22T03:24:02.266145+0000","last_became_peered":"2024-09-22T03:24:02.266145+0000","last_unstale":"2024-09-22T03:24:06.166385+0000","last_undegraded":"2024-09-22T03:24:06.166385+0000","last_fullsized":"2024-09-22T03:24:06.166385+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"161'1947","reported_seq":2657,"reported_epoch":161,"state":"active+recovering","last_fresh":"2024-09-22T03:24:04.108924+0000","last_change":"2024-09-22T03:24:03.148285+0000","last_active":"2024-09-22T03:24:04.108924+0000","last_peered":"2024-09-22T03:24:04.108924+0000","last_clean":"2024-09-22T03:23:20.967292+0000","last_became_active":"2024-09-22T03:24:02.316697+0000","last_became_peered":"2024-09-22T03:24:02.316697+0000","last_unstale":"2024-09-22T03:24:04.108924+0000","last_undegraded":"2024-09-22T03:24:04.108924+0000","last_fullsized":"2024-09-22T03:24:04.108924+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1947,"log_dups_size":0,"ondisk_log_size":1947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11993088,"num_objects":183,"num_object_clones":0,"num_object_copies":549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1947,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"1,2,3","objects":183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":237,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.167664+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:06.167664+0000","last_peered":"2024-09-22T03:24:06.167664+0000","last_clean":"2024-09-22T03:24:06.167664+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:06.167664+0000","last_undegraded":"2024-09-22T03:24:06.167664+0000","last_fullsized":"2024-09-22T03:24:06.167664+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":178913304,"num_objects":2731,"num_object_clones":0,"num_object_copies":8193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":31143,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5659,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33992704,"data_stored":543686728,"data_compressed":25817152,"data_compressed_allocated":33980416,"data_compressed_original":543686656,"omap_allocated":0,"internal_metadata":0},"log_size":31131,"ondisk_log_size":31131,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65128,"kb_used_data":10712,"kb_used_omap":0,"kb_used_meta":54400,"kb_avail":94306712,"statfs":{"total":96636764160,"available":96570073088,"internally_reserved":0,"allocated":10969088,"data_stored":154752292,"data_compressed":7322536,"data_compressed_allocated":9637888,"data_compressed_original":154206208,"omap_allocated":0,"internal_metadata":55705600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":156,"seq":670014898185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52432,"kb_used_data":10944,"kb_used_omap":0,"kb_used_meta":41472,"kb_avail":94319408,"statfs":{"total":96636764160,"available":96583073792,"internally_reserved":0,"allocated":11206656,"data_stored":158032938,"data_compressed":7480100,"data_compressed_allocated":9846784,"data_compressed_original":157466624,"omap_allocated":0,"internal_metadata":42467328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54520,"kb_used_data":6948,"kb_used_omap":0,"kb_used_meta":47552,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":7114752,"data_stored":92500216,"data_compressed":4368100,"data_compressed_allocated":5750784,"data_compressed_original":91930624,"omap_allocated":0,"internal_metadata":48693248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63856,"kb_used_data":11808,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":12091392,"data_stored":172254231,"data_compressed":8155404,"data_compressed_allocated":10735616,"data_compressed_original":171687936,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10153984,"data_stored":162398232,"data_compressed":7711536,"data_compressed_allocated":10149888,"data_compressed_original":162398208,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4866048,"data_stored":77856768,"data_compressed":3697056,"data_compressed_allocated":4866048,"data_compressed_original":77856768,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9789440,"data_stored":156565528,"data_compressed":7434568,"data_compressed_allocated":9785344,"data_compressed_original":156565504,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9183232,"data_stored":146866200,"data_compressed":6973992,"data_compressed_allocated":9179136,"data_compressed_original":146866176,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:09.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:24:09.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:09.148+0000 7f75b3f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:09.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:09.156+0000 7f75b3f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:09.575 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:09.575 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 139/8205 objects degraded (1.694%), 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":794,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":165,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975441,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovering","count":4},{"state_name":"active+recovery_wait+degraded","count":2},{"state_name":"active+clean+snaptrim","count":1},{"state_name":"activating","count":1}],"num_pgs":25,"num_pools":3,"num_objects":2735,"data_bytes":179503691,"bytes_used":241598464,"bytes_avail":386305458176,"bytes_total":386547056640,"inactive_pgs_ratio":0.039999999105930328,"degraded_objects":139,"degraded_total":8205,"degraded_ratio":0.016940889701401583,"recovering_objects_per_sec":102,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":437,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":234},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:12:01.234034+0000","services":{}},"progress_events":{}} 2024-09-22T03:24:09.593 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:24:09.593 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:09.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:09.725+0000 7fe65ac4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:09.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:09.734+0000 7fe65ac4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:09.991 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:09.991 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:10.007 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":551,"stamp":"2024-09-22T03:24:09.391243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":93520459,"num_objects":1423,"num_object_clones":0,"num_object_copies":4269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":32594,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5876,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32539,"ondisk_log_size":32539,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":235612,"kb_used_data":27600,"kb_used_omap":0,"kb_used_meta":207936,"kb_avail":377251748,"statfs":{"total":386547056640,"available":386305789952,"internally_reserved":0,"allocated":28262400,"data_stored":366094629,"data_compressed":17283716,"data_compressed_allocated":22753280,"data_compressed_original":363806720,"omap_allocated":0,"internal_metadata":212926464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-93978624,"num_objects":-1434,"num_object_clones":0,"num_object_copies":-4302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.288561"},"pg_stats":[{"pgid":"5.f","version":"165'1954","reported_seq":2126,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.047731+0000","last_change":"2024-09-22T03:23:56.200281+0000","last_active":"2024-09-22T03:24:09.047731+0000","last_peered":"2024-09-22T03:24:09.047731+0000","last_clean":"2024-09-22T03:24:09.047731+0000","last_became_active":"2024-09-22T03:23:56.190271+0000","last_became_peered":"2024-09-22T03:23:56.190271+0000","last_unstale":"2024-09-22T03:24:09.047731+0000","last_undegraded":"2024-09-22T03:24:09.047731+0000","last_fullsized":"2024-09-22T03:24:09.047731+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1954,"log_dups_size":0,"ondisk_log_size":1954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1955,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"165'2020","reported_seq":2829,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.117752+0000","last_change":"2024-09-22T03:23:56.259934+0000","last_active":"2024-09-22T03:24:09.117752+0000","last_peered":"2024-09-22T03:24:09.117752+0000","last_clean":"2024-09-22T03:24:09.117752+0000","last_became_active":"2024-09-22T03:23:56.182078+0000","last_became_peered":"2024-09-22T03:23:56.182078+0000","last_unstale":"2024-09-22T03:24:09.117752+0000","last_undegraded":"2024-09-22T03:24:09.117752+0000","last_fullsized":"2024-09-22T03:24:09.117752+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2021,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"165'2028","reported_seq":2814,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.137003+0000","last_change":"2024-09-22T03:23:56.218248+0000","last_active":"2024-09-22T03:24:09.137003+0000","last_peered":"2024-09-22T03:24:09.137003+0000","last_clean":"2024-09-22T03:24:09.137003+0000","last_became_active":"2024-09-22T03:23:56.211834+0000","last_became_peered":"2024-09-22T03:23:56.211834+0000","last_unstale":"2024-09-22T03:24:09.137003+0000","last_undegraded":"2024-09-22T03:24:09.137003+0000","last_fullsized":"2024-09-22T03:24:09.137003+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2028,"log_dups_size":0,"ondisk_log_size":2028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291480,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2029,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":196632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"164'2061","reported_seq":2950,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-22T03:24:07.987525+0000","last_change":"2024-09-22T03:24:03.096670+0000","last_active":"2024-09-22T03:24:07.987525+0000","last_peered":"2024-09-22T03:24:07.987525+0000","last_clean":"2024-09-22T03:24:07.987525+0000","last_became_active":"2024-09-22T03:24:02.210115+0000","last_became_peered":"2024-09-22T03:24:02.210115+0000","last_unstale":"2024-09-22T03:24:07.987525+0000","last_undegraded":"2024-09-22T03:24:07.987525+0000","last_fullsized":"2024-09-22T03:24:07.987525+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2061,"log_dups_size":0,"ondisk_log_size":2061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2062,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"165'2023","reported_seq":2353,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.031587+0000","last_change":"2024-09-22T03:24:04.561504+0000","last_active":"2024-09-22T03:24:09.031587+0000","last_peered":"2024-09-22T03:24:09.031587+0000","last_clean":"2024-09-22T03:24:09.031587+0000","last_became_active":"2024-09-22T03:24:02.350737+0000","last_became_peered":"2024-09-22T03:24:02.350737+0000","last_unstale":"2024-09-22T03:24:09.031587+0000","last_undegraded":"2024-09-22T03:24:09.031587+0000","last_fullsized":"2024-09-22T03:24:09.031587+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2023,"log_dups_size":0,"ondisk_log_size":2023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2024,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"165'2036","reported_seq":2205,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.998133+0000","last_change":"2024-09-22T03:23:56.225812+0000","last_active":"2024-09-22T03:24:08.998133+0000","last_peered":"2024-09-22T03:24:08.998133+0000","last_clean":"2024-09-22T03:24:08.998133+0000","last_became_active":"2024-09-22T03:23:56.219474+0000","last_became_peered":"2024-09-22T03:23:56.219474+0000","last_unstale":"2024-09-22T03:24:08.998133+0000","last_undegraded":"2024-09-22T03:24:08.998133+0000","last_fullsized":"2024-09-22T03:24:08.998133+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2036,"log_dups_size":0,"ondisk_log_size":2036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2037,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"163'2022","reported_seq":2798,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:07.025354+0000","last_change":"2024-09-22T03:24:03.164828+0000","last_active":"2024-09-22T03:24:07.025354+0000","last_peered":"2024-09-22T03:24:07.025354+0000","last_clean":"2024-09-22T03:24:07.025354+0000","last_became_active":"2024-09-22T03:24:02.321096+0000","last_became_peered":"2024-09-22T03:24:02.321096+0000","last_unstale":"2024-09-22T03:24:07.025354+0000","last_undegraded":"2024-09-22T03:24:07.025354+0000","last_fullsized":"2024-09-22T03:24:07.025354+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2022,"log_dups_size":0,"ondisk_log_size":2022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2023,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"164'2020","reported_seq":5556,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-22T03:24:07.925748+0000","last_change":"2024-09-22T03:24:05.937268+0000","last_active":"2024-09-22T03:24:07.925748+0000","last_peered":"2024-09-22T03:24:07.925748+0000","last_clean":"2024-09-22T03:24:07.925748+0000","last_became_active":"2024-09-22T03:24:03.217586+0000","last_became_peered":"2024-09-22T03:24:03.217586+0000","last_unstale":"2024-09-22T03:24:07.925748+0000","last_undegraded":"2024-09-22T03:24:07.925748+0000","last_fullsized":"2024-09-22T03:24:07.925748+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2021,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1677,"num_bytes_recovered":19726336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"165'1952","reported_seq":2779,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.837105+0000","last_change":"2024-09-22T03:24:05.374653+0000","last_active":"2024-09-22T03:24:08.837105+0000","last_peered":"2024-09-22T03:24:08.837105+0000","last_clean":"2024-09-22T03:24:08.837105+0000","last_became_active":"2024-09-22T03:24:02.313406+0000","last_became_peered":"2024-09-22T03:24:02.313406+0000","last_unstale":"2024-09-22T03:24:08.837105+0000","last_undegraded":"2024-09-22T03:24:08.837105+0000","last_fullsized":"2024-09-22T03:24:08.837105+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1952,"log_dups_size":0,"ondisk_log_size":1952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1953,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":268,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-22T03:24:07.178697+0000","last_change":"2024-09-22T03:23:57.118798+0000","last_active":"2024-09-22T03:24:07.178697+0000","last_peered":"2024-09-22T03:24:07.178697+0000","last_clean":"2024-09-22T03:24:07.178697+0000","last_became_active":"2024-09-22T03:23:56.167489+0000","last_became_peered":"2024-09-22T03:23:56.167489+0000","last_unstale":"2024-09-22T03:24:07.178697+0000","last_undegraded":"2024-09-22T03:24:07.178697+0000","last_fullsized":"2024-09-22T03:24:07.178697+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"165'2064","reported_seq":2925,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.085148+0000","last_change":"2024-09-22T03:24:06.277559+0000","last_active":"2024-09-22T03:24:09.085148+0000","last_peered":"2024-09-22T03:24:09.085148+0000","last_clean":"2024-09-22T03:24:09.085148+0000","last_became_active":"2024-09-22T03:24:02.263994+0000","last_became_peered":"2024-09-22T03:24:02.263994+0000","last_unstale":"2024-09-22T03:24:09.085148+0000","last_undegraded":"2024-09-22T03:24:09.085148+0000","last_fullsized":"2024-09-22T03:24:09.085148+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2064,"log_dups_size":0,"ondisk_log_size":2064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2065,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":391,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.177222+0000","last_change":"2024-09-22T03:24:03.145088+0000","last_active":"2024-09-22T03:24:08.177222+0000","last_peered":"2024-09-22T03:24:08.177222+0000","last_clean":"2024-09-22T03:24:08.177222+0000","last_became_active":"2024-09-22T03:24:02.330151+0000","last_became_peered":"2024-09-22T03:24:02.330151+0000","last_unstale":"2024-09-22T03:24:08.177222+0000","last_undegraded":"2024-09-22T03:24:08.177222+0000","last_fullsized":"2024-09-22T03:24:08.177222+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"165'2111","reported_seq":2696,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.067192+0000","last_change":"2024-09-22T03:23:56.211523+0000","last_active":"2024-09-22T03:24:09.067192+0000","last_peered":"2024-09-22T03:24:09.067192+0000","last_clean":"2024-09-22T03:24:09.067192+0000","last_became_active":"2024-09-22T03:23:56.208082+0000","last_became_peered":"2024-09-22T03:23:56.208082+0000","last_unstale":"2024-09-22T03:24:09.067192+0000","last_undegraded":"2024-09-22T03:24:09.067192+0000","last_fullsized":"2024-09-22T03:24:09.067192+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2111,"log_dups_size":0,"ondisk_log_size":2111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":439,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.168185+0000","last_change":"2024-09-22T03:24:03.155372+0000","last_active":"2024-09-22T03:24:08.168185+0000","last_peered":"2024-09-22T03:24:08.168185+0000","last_clean":"2024-09-22T03:24:08.168185+0000","last_became_active":"2024-09-22T03:24:02.225075+0000","last_became_peered":"2024-09-22T03:24:02.225075+0000","last_unstale":"2024-09-22T03:24:08.168185+0000","last_undegraded":"2024-09-22T03:24:08.168185+0000","last_fullsized":"2024-09-22T03:24:08.168185+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"165'1992","reported_seq":2856,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.769796+0000","last_change":"2024-09-22T03:24:07.048121+0000","last_active":"2024-09-22T03:24:08.769796+0000","last_peered":"2024-09-22T03:24:08.769796+0000","last_clean":"2024-09-22T03:24:08.769796+0000","last_became_active":"2024-09-22T03:24:02.365735+0000","last_became_peered":"2024-09-22T03:24:02.365735+0000","last_unstale":"2024-09-22T03:24:08.769796+0000","last_undegraded":"2024-09-22T03:24:08.769796+0000","last_fullsized":"2024-09-22T03:24:08.769796+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1992,"log_dups_size":0,"ondisk_log_size":1992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1993,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":350,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-22T03:24:07.180427+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:07.180427+0000","last_peered":"2024-09-22T03:24:07.180427+0000","last_clean":"2024-09-22T03:24:07.180427+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:07.180427+0000","last_undegraded":"2024-09-22T03:24:07.180427+0000","last_fullsized":"2024-09-22T03:24:07.180427+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":510,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.176675+0000","last_change":"2024-09-22T03:24:02.377618+0000","last_active":"2024-09-22T03:24:08.176675+0000","last_peered":"2024-09-22T03:24:08.176675+0000","last_clean":"2024-09-22T03:24:08.176675+0000","last_became_active":"2024-09-22T03:24:02.362178+0000","last_became_peered":"2024-09-22T03:24:02.362178+0000","last_unstale":"2024-09-22T03:24:08.176675+0000","last_undegraded":"2024-09-22T03:24:08.176675+0000","last_fullsized":"2024-09-22T03:24:08.176675+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"165'2128","reported_seq":3059,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.216833+0000","last_change":"2024-09-22T03:24:05.400340+0000","last_active":"2024-09-22T03:24:09.216833+0000","last_peered":"2024-09-22T03:24:09.216833+0000","last_clean":"2024-09-22T03:24:09.216833+0000","last_became_active":"2024-09-22T03:24:02.328698+0000","last_became_peered":"2024-09-22T03:24:02.328698+0000","last_unstale":"2024-09-22T03:24:09.216833+0000","last_undegraded":"2024-09-22T03:24:09.216833+0000","last_fullsized":"2024-09-22T03:24:09.216833+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2128,"log_dups_size":0,"ondisk_log_size":2128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2129,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":164,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.169093+0000","last_change":"2024-09-22T03:24:03.169310+0000","last_active":"2024-09-22T03:24:06.169093+0000","last_peered":"2024-09-22T03:24:06.169093+0000","last_clean":"2024-09-22T03:24:06.169093+0000","last_became_active":"2024-09-22T03:24:02.324336+0000","last_became_peered":"2024-09-22T03:24:02.324336+0000","last_unstale":"2024-09-22T03:24:06.169093+0000","last_undegraded":"2024-09-22T03:24:06.169093+0000","last_fullsized":"2024-09-22T03:24:06.169093+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"164'2102","reported_seq":2885,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.039541+0000","last_change":"2024-09-22T03:23:56.170283+0000","last_active":"2024-09-22T03:24:08.039541+0000","last_peered":"2024-09-22T03:24:08.039541+0000","last_clean":"2024-09-22T03:24:08.039541+0000","last_became_active":"2024-09-22T03:23:56.158439+0000","last_became_peered":"2024-09-22T03:23:56.158439+0000","last_unstale":"2024-09-22T03:24:08.039541+0000","last_undegraded":"2024-09-22T03:24:08.039541+0000","last_fullsized":"2024-09-22T03:24:08.039541+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2102,"log_dups_size":0,"ondisk_log_size":2102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2103,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":348,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.167517+0000","last_change":"2024-09-22T03:23:57.117265+0000","last_active":"2024-09-22T03:24:08.167517+0000","last_peered":"2024-09-22T03:24:08.167517+0000","last_clean":"2024-09-22T03:24:08.167517+0000","last_became_active":"2024-09-22T03:23:56.204767+0000","last_became_peered":"2024-09-22T03:23:56.204767+0000","last_unstale":"2024-09-22T03:24:08.167517+0000","last_undegraded":"2024-09-22T03:24:08.167517+0000","last_fullsized":"2024-09-22T03:24:08.167517+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"165'1880","reported_seq":2179,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.219945+0000","last_change":"2024-09-22T03:24:03.658213+0000","last_active":"2024-09-22T03:24:09.219945+0000","last_peered":"2024-09-22T03:24:09.219945+0000","last_clean":"2024-09-22T03:24:09.219945+0000","last_became_active":"2024-09-22T03:24:02.289524+0000","last_became_peered":"2024-09-22T03:24:02.289524+0000","last_unstale":"2024-09-22T03:24:09.219945+0000","last_undegraded":"2024-09-22T03:24:09.219945+0000","last_fullsized":"2024-09-22T03:24:09.219945+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1880,"log_dups_size":0,"ondisk_log_size":1880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1881,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":282,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.166385+0000","last_change":"2024-09-22T03:24:03.155475+0000","last_active":"2024-09-22T03:24:06.166385+0000","last_peered":"2024-09-22T03:24:06.166385+0000","last_clean":"2024-09-22T03:24:06.166385+0000","last_became_active":"2024-09-22T03:24:02.266145+0000","last_became_peered":"2024-09-22T03:24:02.266145+0000","last_unstale":"2024-09-22T03:24:06.166385+0000","last_undegraded":"2024-09-22T03:24:06.166385+0000","last_fullsized":"2024-09-22T03:24:06.166385+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"165'2046","reported_seq":2797,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.936769+0000","last_change":"2024-09-22T03:24:04.680619+0000","last_active":"2024-09-22T03:24:08.936769+0000","last_peered":"2024-09-22T03:24:08.936769+0000","last_clean":"2024-09-22T03:24:08.936769+0000","last_became_active":"2024-09-22T03:24:02.316697+0000","last_became_peered":"2024-09-22T03:24:02.316697+0000","last_unstale":"2024-09-22T03:24:08.936769+0000","last_undegraded":"2024-09-22T03:24:08.936769+0000","last_fullsized":"2024-09-22T03:24:08.936769+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2046,"log_dups_size":0,"ondisk_log_size":2046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":237,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.167664+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:06.167664+0000","last_peered":"2024-09-22T03:24:06.167664+0000","last_clean":"2024-09-22T03:24:06.167664+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:06.167664+0000","last_undegraded":"2024-09-22T03:24:06.167664+0000","last_fullsized":"2024-09-22T03:24:06.167664+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":92930072,"num_objects":1419,"num_object_clones":0,"num_object_copies":4257,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":32455,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5876,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20541440,"data_stored":328466504,"data_compressed":15597344,"data_compressed_allocated":20529152,"data_compressed_original":328466432,"omap_allocated":0,"internal_metadata":0},"log_size":32439,"ondisk_log_size":32439,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65544,"kb_used_data":6968,"kb_used_omap":0,"kb_used_meta":58560,"kb_avail":94306296,"statfs":{"total":96636764160,"available":96569647104,"internally_reserved":0,"allocated":7135232,"data_stored":92899516,"data_compressed":4384808,"data_compressed_allocated":5771264,"data_compressed_original":92340224,"omap_allocated":0,"internal_metadata":59965440},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":156,"seq":670014898190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51704,"kb_used_data":6052,"kb_used_omap":0,"kb_used_meta":45632,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":6197248,"data_stored":77371330,"data_compressed":3649228,"data_compressed_allocated":4804608,"data_compressed_original":76791808,"omap_allocated":0,"internal_metadata":46727168},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54520,"kb_used_data":6948,"kb_used_omap":0,"kb_used_meta":47552,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":7114752,"data_stored":92500216,"data_compressed":4368100,"data_compressed_allocated":5750784,"data_compressed_original":91930624,"omap_allocated":0,"internal_metadata":48693248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63844,"kb_used_data":7632,"kb_used_omap":0,"kb_used_meta":56192,"kb_avail":94307996,"statfs":{"total":96636764160,"available":96571387904,"internally_reserved":0,"allocated":7815168,"data_stored":103323567,"data_compressed":4881580,"data_compressed_allocated":6426624,"data_compressed_original":102744064,"omap_allocated":0,"internal_metadata":57540608},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5873664,"data_stored":93913112,"data_compressed":4459496,"data_compressed_allocated":5869568,"data_compressed_original":93913088,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4866048,"data_stored":77856768,"data_compressed":3697056,"data_compressed_allocated":4866048,"data_compressed_original":77856768,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4767744,"data_stored":76218392,"data_compressed":3619256,"data_compressed_allocated":4763648,"data_compressed_original":76218368,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":80478232,"data_compressed":3821536,"data_compressed_allocated":5029888,"data_compressed_original":80478208,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:11.262 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:11.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:11.399+0000 7f5a7e557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:11.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:11.405+0000 7f5a7e557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:13.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:13.127+0000 7f2665a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:13.137+0000 7f2665a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:13.402 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:13.414 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-09-22T03:24:13.391975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70779467,"num_objects":1076,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":32941,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5876,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32886,"ondisk_log_size":32886,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":231524,"kb_used_data":19356,"kb_used_omap":0,"kb_used_meta":212096,"kb_avail":377255836,"statfs":{"total":386547056640,"available":386309976064,"internally_reserved":0,"allocated":19820544,"data_stored":230258297,"data_compressed":10832540,"data_compressed_allocated":14262272,"data_compressed_original":227950592,"omap_allocated":0,"internal_metadata":217186304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-108724224,"num_objects":-1659,"num_object_clones":0,"num_object_copies":-4977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1659,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.251924"},"pg_stats":[{"pgid":"5.f","version":"165'1954","reported_seq":2126,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.047731+0000","last_change":"2024-09-22T03:23:56.200281+0000","last_active":"2024-09-22T03:24:09.047731+0000","last_peered":"2024-09-22T03:24:09.047731+0000","last_clean":"2024-09-22T03:24:09.047731+0000","last_became_active":"2024-09-22T03:23:56.190271+0000","last_became_peered":"2024-09-22T03:23:56.190271+0000","last_unstale":"2024-09-22T03:24:09.047731+0000","last_undegraded":"2024-09-22T03:24:09.047731+0000","last_fullsized":"2024-09-22T03:24:09.047731+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1954,"log_dups_size":0,"ondisk_log_size":1954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1955,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"165'2020","reported_seq":2829,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.117752+0000","last_change":"2024-09-22T03:23:56.259934+0000","last_active":"2024-09-22T03:24:09.117752+0000","last_peered":"2024-09-22T03:24:09.117752+0000","last_clean":"2024-09-22T03:24:09.117752+0000","last_became_active":"2024-09-22T03:23:56.182078+0000","last_became_peered":"2024-09-22T03:23:56.182078+0000","last_unstale":"2024-09-22T03:24:09.117752+0000","last_undegraded":"2024-09-22T03:24:09.117752+0000","last_fullsized":"2024-09-22T03:24:09.117752+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2021,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"165'2028","reported_seq":2814,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.137003+0000","last_change":"2024-09-22T03:23:56.218248+0000","last_active":"2024-09-22T03:24:09.137003+0000","last_peered":"2024-09-22T03:24:09.137003+0000","last_clean":"2024-09-22T03:24:09.137003+0000","last_became_active":"2024-09-22T03:23:56.211834+0000","last_became_peered":"2024-09-22T03:23:56.211834+0000","last_unstale":"2024-09-22T03:24:09.137003+0000","last_undegraded":"2024-09-22T03:24:09.137003+0000","last_fullsized":"2024-09-22T03:24:09.137003+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2028,"log_dups_size":0,"ondisk_log_size":2028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291480,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2029,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":196632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"166'2141","reported_seq":3036,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:12.935553+0000","last_change":"2024-09-22T03:24:03.096670+0000","last_active":"2024-09-22T03:24:12.935553+0000","last_peered":"2024-09-22T03:24:12.935553+0000","last_clean":"2024-09-22T03:24:12.935553+0000","last_became_active":"2024-09-22T03:24:02.210115+0000","last_became_peered":"2024-09-22T03:24:02.210115+0000","last_unstale":"2024-09-22T03:24:12.935553+0000","last_undegraded":"2024-09-22T03:24:12.935553+0000","last_fullsized":"2024-09-22T03:24:12.935553+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2141,"log_dups_size":0,"ondisk_log_size":2141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2142,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"165'2023","reported_seq":2353,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.031587+0000","last_change":"2024-09-22T03:24:04.561504+0000","last_active":"2024-09-22T03:24:09.031587+0000","last_peered":"2024-09-22T03:24:09.031587+0000","last_clean":"2024-09-22T03:24:09.031587+0000","last_became_active":"2024-09-22T03:24:02.350737+0000","last_became_peered":"2024-09-22T03:24:02.350737+0000","last_unstale":"2024-09-22T03:24:09.031587+0000","last_undegraded":"2024-09-22T03:24:09.031587+0000","last_fullsized":"2024-09-22T03:24:09.031587+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2023,"log_dups_size":0,"ondisk_log_size":2023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2024,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"165'2036","reported_seq":2205,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.998133+0000","last_change":"2024-09-22T03:23:56.225812+0000","last_active":"2024-09-22T03:24:08.998133+0000","last_peered":"2024-09-22T03:24:08.998133+0000","last_clean":"2024-09-22T03:24:08.998133+0000","last_became_active":"2024-09-22T03:23:56.219474+0000","last_became_peered":"2024-09-22T03:23:56.219474+0000","last_unstale":"2024-09-22T03:24:08.998133+0000","last_undegraded":"2024-09-22T03:24:08.998133+0000","last_fullsized":"2024-09-22T03:24:08.998133+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2036,"log_dups_size":0,"ondisk_log_size":2036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2037,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"166'2103","reported_seq":2888,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:12.080250+0000","last_change":"2024-09-22T03:24:03.164828+0000","last_active":"2024-09-22T03:24:12.080250+0000","last_peered":"2024-09-22T03:24:12.080250+0000","last_clean":"2024-09-22T03:24:12.080250+0000","last_became_active":"2024-09-22T03:24:02.321096+0000","last_became_peered":"2024-09-22T03:24:02.321096+0000","last_unstale":"2024-09-22T03:24:12.080250+0000","last_undegraded":"2024-09-22T03:24:12.080250+0000","last_fullsized":"2024-09-22T03:24:12.080250+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2103,"log_dups_size":0,"ondisk_log_size":2103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2104,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"166'2116","reported_seq":5658,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:12.996463+0000","last_change":"2024-09-22T03:24:05.937268+0000","last_active":"2024-09-22T03:24:12.996463+0000","last_peered":"2024-09-22T03:24:12.996463+0000","last_clean":"2024-09-22T03:24:12.996463+0000","last_became_active":"2024-09-22T03:24:03.217586+0000","last_became_peered":"2024-09-22T03:24:03.217586+0000","last_unstale":"2024-09-22T03:24:12.996463+0000","last_undegraded":"2024-09-22T03:24:12.996463+0000","last_fullsized":"2024-09-22T03:24:12.996463+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2117,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1677,"num_bytes_recovered":19726336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"165'1952","reported_seq":2779,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.837105+0000","last_change":"2024-09-22T03:24:05.374653+0000","last_active":"2024-09-22T03:24:08.837105+0000","last_peered":"2024-09-22T03:24:08.837105+0000","last_clean":"2024-09-22T03:24:08.837105+0000","last_became_active":"2024-09-22T03:24:02.313406+0000","last_became_peered":"2024-09-22T03:24:02.313406+0000","last_unstale":"2024-09-22T03:24:08.837105+0000","last_undegraded":"2024-09-22T03:24:08.837105+0000","last_fullsized":"2024-09-22T03:24:08.837105+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1952,"log_dups_size":0,"ondisk_log_size":1952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1953,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":274,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.125765+0000","last_change":"2024-09-22T03:23:57.118798+0000","last_active":"2024-09-22T03:24:10.125765+0000","last_peered":"2024-09-22T03:24:10.125765+0000","last_clean":"2024-09-22T03:24:10.125765+0000","last_became_active":"2024-09-22T03:23:56.167489+0000","last_became_peered":"2024-09-22T03:23:56.167489+0000","last_unstale":"2024-09-22T03:24:10.125765+0000","last_undegraded":"2024-09-22T03:24:10.125765+0000","last_fullsized":"2024-09-22T03:24:10.125765+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"165'2064","reported_seq":2925,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.085148+0000","last_change":"2024-09-22T03:24:06.277559+0000","last_active":"2024-09-22T03:24:09.085148+0000","last_peered":"2024-09-22T03:24:09.085148+0000","last_clean":"2024-09-22T03:24:09.085148+0000","last_became_active":"2024-09-22T03:24:02.263994+0000","last_became_peered":"2024-09-22T03:24:02.263994+0000","last_unstale":"2024-09-22T03:24:09.085148+0000","last_undegraded":"2024-09-22T03:24:09.085148+0000","last_fullsized":"2024-09-22T03:24:09.085148+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2064,"log_dups_size":0,"ondisk_log_size":2064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2065,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":391,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.177222+0000","last_change":"2024-09-22T03:24:03.145088+0000","last_active":"2024-09-22T03:24:08.177222+0000","last_peered":"2024-09-22T03:24:08.177222+0000","last_clean":"2024-09-22T03:24:08.177222+0000","last_became_active":"2024-09-22T03:24:02.330151+0000","last_became_peered":"2024-09-22T03:24:02.330151+0000","last_unstale":"2024-09-22T03:24:08.177222+0000","last_undegraded":"2024-09-22T03:24:08.177222+0000","last_fullsized":"2024-09-22T03:24:08.177222+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"165'2111","reported_seq":2696,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.067192+0000","last_change":"2024-09-22T03:23:56.211523+0000","last_active":"2024-09-22T03:24:09.067192+0000","last_peered":"2024-09-22T03:24:09.067192+0000","last_clean":"2024-09-22T03:24:09.067192+0000","last_became_active":"2024-09-22T03:23:56.208082+0000","last_became_peered":"2024-09-22T03:23:56.208082+0000","last_unstale":"2024-09-22T03:24:09.067192+0000","last_undegraded":"2024-09-22T03:24:09.067192+0000","last_fullsized":"2024-09-22T03:24:09.067192+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2111,"log_dups_size":0,"ondisk_log_size":2111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":439,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.168185+0000","last_change":"2024-09-22T03:24:03.155372+0000","last_active":"2024-09-22T03:24:08.168185+0000","last_peered":"2024-09-22T03:24:08.168185+0000","last_clean":"2024-09-22T03:24:08.168185+0000","last_became_active":"2024-09-22T03:24:02.225075+0000","last_became_peered":"2024-09-22T03:24:02.225075+0000","last_unstale":"2024-09-22T03:24:08.168185+0000","last_undegraded":"2024-09-22T03:24:08.168185+0000","last_fullsized":"2024-09-22T03:24:08.168185+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"165'1992","reported_seq":2856,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.769796+0000","last_change":"2024-09-22T03:24:07.048121+0000","last_active":"2024-09-22T03:24:08.769796+0000","last_peered":"2024-09-22T03:24:08.769796+0000","last_clean":"2024-09-22T03:24:08.769796+0000","last_became_active":"2024-09-22T03:24:02.365735+0000","last_became_peered":"2024-09-22T03:24:02.365735+0000","last_unstale":"2024-09-22T03:24:08.769796+0000","last_undegraded":"2024-09-22T03:24:08.769796+0000","last_fullsized":"2024-09-22T03:24:08.769796+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1992,"log_dups_size":0,"ondisk_log_size":1992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1993,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":356,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.128075+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:10.128075+0000","last_peered":"2024-09-22T03:24:10.128075+0000","last_clean":"2024-09-22T03:24:10.128075+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:10.128075+0000","last_undegraded":"2024-09-22T03:24:10.128075+0000","last_fullsized":"2024-09-22T03:24:10.128075+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":510,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.176675+0000","last_change":"2024-09-22T03:24:02.377618+0000","last_active":"2024-09-22T03:24:08.176675+0000","last_peered":"2024-09-22T03:24:08.176675+0000","last_clean":"2024-09-22T03:24:08.176675+0000","last_became_active":"2024-09-22T03:24:02.362178+0000","last_became_peered":"2024-09-22T03:24:02.362178+0000","last_unstale":"2024-09-22T03:24:08.176675+0000","last_undegraded":"2024-09-22T03:24:08.176675+0000","last_fullsized":"2024-09-22T03:24:08.176675+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"165'2128","reported_seq":3059,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.216833+0000","last_change":"2024-09-22T03:24:05.400340+0000","last_active":"2024-09-22T03:24:09.216833+0000","last_peered":"2024-09-22T03:24:09.216833+0000","last_clean":"2024-09-22T03:24:09.216833+0000","last_became_active":"2024-09-22T03:24:02.328698+0000","last_became_peered":"2024-09-22T03:24:02.328698+0000","last_unstale":"2024-09-22T03:24:09.216833+0000","last_undegraded":"2024-09-22T03:24:09.216833+0000","last_fullsized":"2024-09-22T03:24:09.216833+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2128,"log_dups_size":0,"ondisk_log_size":2128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2129,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":173,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.130663+0000","last_change":"2024-09-22T03:24:03.169310+0000","last_active":"2024-09-22T03:24:10.130663+0000","last_peered":"2024-09-22T03:24:10.130663+0000","last_clean":"2024-09-22T03:24:10.130663+0000","last_became_active":"2024-09-22T03:24:02.324336+0000","last_became_peered":"2024-09-22T03:24:02.324336+0000","last_unstale":"2024-09-22T03:24:10.130663+0000","last_undegraded":"2024-09-22T03:24:10.130663+0000","last_fullsized":"2024-09-22T03:24:10.130663+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"166'2192","reported_seq":2981,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:13.100095+0000","last_change":"2024-09-22T03:23:56.170283+0000","last_active":"2024-09-22T03:24:13.100095+0000","last_peered":"2024-09-22T03:24:13.100095+0000","last_clean":"2024-09-22T03:24:13.100095+0000","last_became_active":"2024-09-22T03:23:56.158439+0000","last_became_peered":"2024-09-22T03:23:56.158439+0000","last_unstale":"2024-09-22T03:24:13.100095+0000","last_undegraded":"2024-09-22T03:24:13.100095+0000","last_fullsized":"2024-09-22T03:24:13.100095+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2193,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":348,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.167517+0000","last_change":"2024-09-22T03:23:57.117265+0000","last_active":"2024-09-22T03:24:08.167517+0000","last_peered":"2024-09-22T03:24:08.167517+0000","last_clean":"2024-09-22T03:24:08.167517+0000","last_became_active":"2024-09-22T03:23:56.204767+0000","last_became_peered":"2024-09-22T03:23:56.204767+0000","last_unstale":"2024-09-22T03:24:08.167517+0000","last_undegraded":"2024-09-22T03:24:08.167517+0000","last_fullsized":"2024-09-22T03:24:08.167517+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"165'1880","reported_seq":2179,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.219945+0000","last_change":"2024-09-22T03:24:03.658213+0000","last_active":"2024-09-22T03:24:09.219945+0000","last_peered":"2024-09-22T03:24:09.219945+0000","last_clean":"2024-09-22T03:24:09.219945+0000","last_became_active":"2024-09-22T03:24:02.289524+0000","last_became_peered":"2024-09-22T03:24:02.289524+0000","last_unstale":"2024-09-22T03:24:09.219945+0000","last_undegraded":"2024-09-22T03:24:09.219945+0000","last_fullsized":"2024-09-22T03:24:09.219945+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1880,"log_dups_size":0,"ondisk_log_size":1880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1881,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":291,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.129676+0000","last_change":"2024-09-22T03:24:03.155475+0000","last_active":"2024-09-22T03:24:10.129676+0000","last_peered":"2024-09-22T03:24:10.129676+0000","last_clean":"2024-09-22T03:24:10.129676+0000","last_became_active":"2024-09-22T03:24:02.266145+0000","last_became_peered":"2024-09-22T03:24:02.266145+0000","last_unstale":"2024-09-22T03:24:10.129676+0000","last_undegraded":"2024-09-22T03:24:10.129676+0000","last_fullsized":"2024-09-22T03:24:10.129676+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"165'2046","reported_seq":2797,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.936769+0000","last_change":"2024-09-22T03:24:04.680619+0000","last_active":"2024-09-22T03:24:08.936769+0000","last_peered":"2024-09-22T03:24:08.936769+0000","last_clean":"2024-09-22T03:24:08.936769+0000","last_became_active":"2024-09-22T03:24:02.316697+0000","last_became_peered":"2024-09-22T03:24:02.316697+0000","last_unstale":"2024-09-22T03:24:08.936769+0000","last_undegraded":"2024-09-22T03:24:08.936769+0000","last_fullsized":"2024-09-22T03:24:08.936769+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2046,"log_dups_size":0,"ondisk_log_size":2046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":246,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.130829+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:10.130829+0000","last_peered":"2024-09-22T03:24:10.130829+0000","last_clean":"2024-09-22T03:24:10.130829+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:10.130829+0000","last_undegraded":"2024-09-22T03:24:10.130829+0000","last_fullsized":"2024-09-22T03:24:10.130829+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":70189080,"num_objects":1072,"num_object_clones":0,"num_object_copies":3216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":32802,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5876,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12132352,"data_stored":193921096,"data_compressed":9208408,"data_compressed_allocated":12120064,"data_compressed_original":193921024,"omap_allocated":0,"internal_metadata":0},"log_size":32786,"ondisk_log_size":32786,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65224,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":62720,"kb_avail":94306616,"statfs":{"total":96636764160,"available":96569974784,"internally_reserved":0,"allocated":2547712,"data_stored":19243648,"data_compressed":886920,"data_compressed_allocated":1167360,"data_compressed_original":18677760,"omap_allocated":0,"internal_metadata":64225280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":156,"seq":670014898190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51704,"kb_used_data":6052,"kb_used_omap":0,"kb_used_meta":45632,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":6197248,"data_stored":77371330,"data_compressed":3649228,"data_compressed_allocated":4804608,"data_compressed_original":76791808,"omap_allocated":0,"internal_metadata":46727168},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50752,"kb_used_data":3184,"kb_used_omap":0,"kb_used_meta":47552,"kb_avail":94321088,"statfs":{"total":96636764160,"available":96584794112,"internally_reserved":0,"allocated":3260416,"data_stored":30319752,"data_compressed":1414812,"data_compressed_allocated":1863680,"data_compressed_original":29736960,"omap_allocated":0,"internal_metadata":48693248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63844,"kb_used_data":7632,"kb_used_omap":0,"kb_used_meta":56192,"kb_avail":94307996,"statfs":{"total":96636764160,"available":96571387904,"internally_reserved":0,"allocated":7815168,"data_stored":103323567,"data_compressed":4881580,"data_compressed_allocated":6426624,"data_compressed_original":102744064,"omap_allocated":0,"internal_metadata":57540608},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5873664,"data_stored":93913112,"data_compressed":4459496,"data_compressed_allocated":5869568,"data_compressed_original":93913088,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1081344,"data_stored":17301504,"data_compressed":821568,"data_compressed_allocated":1081344,"data_compressed_original":17301504,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4767744,"data_stored":76218392,"data_compressed":3619256,"data_compressed_allocated":4763648,"data_compressed_original":76218368,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6488088,"data_compressed":308088,"data_compressed_allocated":405504,"data_compressed_original":6488064,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:13.415 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:24:13.415 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:24:13.416 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:13.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:13.535+0000 7f1fd8397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:13.544+0000 7f1fd8397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.760 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Removed 16936 objects 2024-09-22T03:24:13.760 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Clean up completed and total clean up time :127.243 2024-09-22T03:24:13.802 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:13.802 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:13.814 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-09-22T03:24:13.391975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70779467,"num_objects":1076,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":32941,"num_write_kb":1085801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5876,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32886,"ondisk_log_size":32886,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":231524,"kb_used_data":19356,"kb_used_omap":0,"kb_used_meta":212096,"kb_avail":377255836,"statfs":{"total":386547056640,"available":386309976064,"internally_reserved":0,"allocated":19820544,"data_stored":230258297,"data_compressed":10832540,"data_compressed_allocated":14262272,"data_compressed_original":227950592,"omap_allocated":0,"internal_metadata":217186304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-108724224,"num_objects":-1659,"num_object_clones":0,"num_object_copies":-4977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1659,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.251924"},"pg_stats":[{"pgid":"5.f","version":"165'1954","reported_seq":2126,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.047731+0000","last_change":"2024-09-22T03:23:56.200281+0000","last_active":"2024-09-22T03:24:09.047731+0000","last_peered":"2024-09-22T03:24:09.047731+0000","last_clean":"2024-09-22T03:24:09.047731+0000","last_became_active":"2024-09-22T03:23:56.190271+0000","last_became_peered":"2024-09-22T03:23:56.190271+0000","last_unstale":"2024-09-22T03:24:09.047731+0000","last_undegraded":"2024-09-22T03:24:09.047731+0000","last_fullsized":"2024-09-22T03:24:09.047731+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1954,"log_dups_size":0,"ondisk_log_size":1954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1955,"num_write_kb":65472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"165'2020","reported_seq":2829,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.117752+0000","last_change":"2024-09-22T03:23:56.259934+0000","last_active":"2024-09-22T03:24:09.117752+0000","last_peered":"2024-09-22T03:24:09.117752+0000","last_clean":"2024-09-22T03:24:09.117752+0000","last_became_active":"2024-09-22T03:23:56.182078+0000","last_became_peered":"2024-09-22T03:23:56.182078+0000","last_unstale":"2024-09-22T03:24:09.117752+0000","last_undegraded":"2024-09-22T03:24:09.117752+0000","last_fullsized":"2024-09-22T03:24:09.117752+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2021,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"165'2028","reported_seq":2814,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.137003+0000","last_change":"2024-09-22T03:23:56.218248+0000","last_active":"2024-09-22T03:24:09.137003+0000","last_peered":"2024-09-22T03:24:09.137003+0000","last_clean":"2024-09-22T03:24:09.137003+0000","last_became_active":"2024-09-22T03:23:56.211834+0000","last_became_peered":"2024-09-22T03:23:56.211834+0000","last_unstale":"2024-09-22T03:24:09.137003+0000","last_undegraded":"2024-09-22T03:24:09.137003+0000","last_fullsized":"2024-09-22T03:24:09.137003+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2028,"log_dups_size":0,"ondisk_log_size":2028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291480,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2029,"num_write_kb":67969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":196632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"166'2141","reported_seq":3036,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:12.935553+0000","last_change":"2024-09-22T03:24:03.096670+0000","last_active":"2024-09-22T03:24:12.935553+0000","last_peered":"2024-09-22T03:24:12.935553+0000","last_clean":"2024-09-22T03:24:12.935553+0000","last_became_active":"2024-09-22T03:24:02.210115+0000","last_became_peered":"2024-09-22T03:24:02.210115+0000","last_unstale":"2024-09-22T03:24:12.935553+0000","last_undegraded":"2024-09-22T03:24:12.935553+0000","last_fullsized":"2024-09-22T03:24:12.935553+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2141,"log_dups_size":0,"ondisk_log_size":2141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2142,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"165'2023","reported_seq":2353,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.031587+0000","last_change":"2024-09-22T03:24:04.561504+0000","last_active":"2024-09-22T03:24:09.031587+0000","last_peered":"2024-09-22T03:24:09.031587+0000","last_clean":"2024-09-22T03:24:09.031587+0000","last_became_active":"2024-09-22T03:24:02.350737+0000","last_became_peered":"2024-09-22T03:24:02.350737+0000","last_unstale":"2024-09-22T03:24:09.031587+0000","last_undegraded":"2024-09-22T03:24:09.031587+0000","last_fullsized":"2024-09-22T03:24:09.031587+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2023,"log_dups_size":0,"ondisk_log_size":2023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2024,"num_write_kb":67328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"165'2036","reported_seq":2205,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.998133+0000","last_change":"2024-09-22T03:23:56.225812+0000","last_active":"2024-09-22T03:24:08.998133+0000","last_peered":"2024-09-22T03:24:08.998133+0000","last_clean":"2024-09-22T03:24:08.998133+0000","last_became_active":"2024-09-22T03:23:56.219474+0000","last_became_peered":"2024-09-22T03:23:56.219474+0000","last_unstale":"2024-09-22T03:24:08.998133+0000","last_undegraded":"2024-09-22T03:24:08.998133+0000","last_fullsized":"2024-09-22T03:24:08.998133+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2036,"log_dups_size":0,"ondisk_log_size":2036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2037,"num_write_kb":67840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"166'2103","reported_seq":2888,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:12.080250+0000","last_change":"2024-09-22T03:24:03.164828+0000","last_active":"2024-09-22T03:24:12.080250+0000","last_peered":"2024-09-22T03:24:12.080250+0000","last_clean":"2024-09-22T03:24:12.080250+0000","last_became_active":"2024-09-22T03:24:02.321096+0000","last_became_peered":"2024-09-22T03:24:02.321096+0000","last_unstale":"2024-09-22T03:24:12.080250+0000","last_undegraded":"2024-09-22T03:24:12.080250+0000","last_fullsized":"2024-09-22T03:24:12.080250+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2103,"log_dups_size":0,"ondisk_log_size":2103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2104,"num_write_kb":68480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"166'2116","reported_seq":5658,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:12.996463+0000","last_change":"2024-09-22T03:24:05.937268+0000","last_active":"2024-09-22T03:24:12.996463+0000","last_peered":"2024-09-22T03:24:12.996463+0000","last_clean":"2024-09-22T03:24:12.996463+0000","last_became_active":"2024-09-22T03:24:03.217586+0000","last_became_peered":"2024-09-22T03:24:03.217586+0000","last_unstale":"2024-09-22T03:24:12.996463+0000","last_undegraded":"2024-09-22T03:24:12.996463+0000","last_fullsized":"2024-09-22T03:24:12.996463+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2117,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1677,"num_bytes_recovered":19726336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"165'1952","reported_seq":2779,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.837105+0000","last_change":"2024-09-22T03:24:05.374653+0000","last_active":"2024-09-22T03:24:08.837105+0000","last_peered":"2024-09-22T03:24:08.837105+0000","last_clean":"2024-09-22T03:24:08.837105+0000","last_became_active":"2024-09-22T03:24:02.313406+0000","last_became_peered":"2024-09-22T03:24:02.313406+0000","last_unstale":"2024-09-22T03:24:08.837105+0000","last_undegraded":"2024-09-22T03:24:08.837105+0000","last_fullsized":"2024-09-22T03:24:08.837105+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1952,"log_dups_size":0,"ondisk_log_size":1952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1953,"num_write_kb":65280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":274,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.125765+0000","last_change":"2024-09-22T03:23:57.118798+0000","last_active":"2024-09-22T03:24:10.125765+0000","last_peered":"2024-09-22T03:24:10.125765+0000","last_clean":"2024-09-22T03:24:10.125765+0000","last_became_active":"2024-09-22T03:23:56.167489+0000","last_became_peered":"2024-09-22T03:23:56.167489+0000","last_unstale":"2024-09-22T03:24:10.125765+0000","last_undegraded":"2024-09-22T03:24:10.125765+0000","last_fullsized":"2024-09-22T03:24:10.125765+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"165'2064","reported_seq":2925,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.085148+0000","last_change":"2024-09-22T03:24:06.277559+0000","last_active":"2024-09-22T03:24:09.085148+0000","last_peered":"2024-09-22T03:24:09.085148+0000","last_clean":"2024-09-22T03:24:09.085148+0000","last_became_active":"2024-09-22T03:24:02.263994+0000","last_became_peered":"2024-09-22T03:24:02.263994+0000","last_unstale":"2024-09-22T03:24:09.085148+0000","last_undegraded":"2024-09-22T03:24:09.085148+0000","last_fullsized":"2024-09-22T03:24:09.085148+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2064,"log_dups_size":0,"ondisk_log_size":2064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2065,"num_write_kb":68992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":391,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.177222+0000","last_change":"2024-09-22T03:24:03.145088+0000","last_active":"2024-09-22T03:24:08.177222+0000","last_peered":"2024-09-22T03:24:08.177222+0000","last_clean":"2024-09-22T03:24:08.177222+0000","last_became_active":"2024-09-22T03:24:02.330151+0000","last_became_peered":"2024-09-22T03:24:02.330151+0000","last_unstale":"2024-09-22T03:24:08.177222+0000","last_undegraded":"2024-09-22T03:24:08.177222+0000","last_fullsized":"2024-09-22T03:24:08.177222+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"165'2111","reported_seq":2696,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.067192+0000","last_change":"2024-09-22T03:23:56.211523+0000","last_active":"2024-09-22T03:24:09.067192+0000","last_peered":"2024-09-22T03:24:09.067192+0000","last_clean":"2024-09-22T03:24:09.067192+0000","last_became_active":"2024-09-22T03:23:56.208082+0000","last_became_peered":"2024-09-22T03:23:56.208082+0000","last_unstale":"2024-09-22T03:24:09.067192+0000","last_undegraded":"2024-09-22T03:24:09.067192+0000","last_fullsized":"2024-09-22T03:24:09.067192+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2111,"log_dups_size":0,"ondisk_log_size":2111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":70080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":439,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.168185+0000","last_change":"2024-09-22T03:24:03.155372+0000","last_active":"2024-09-22T03:24:08.168185+0000","last_peered":"2024-09-22T03:24:08.168185+0000","last_clean":"2024-09-22T03:24:08.168185+0000","last_became_active":"2024-09-22T03:24:02.225075+0000","last_became_peered":"2024-09-22T03:24:02.225075+0000","last_unstale":"2024-09-22T03:24:08.168185+0000","last_undegraded":"2024-09-22T03:24:08.168185+0000","last_fullsized":"2024-09-22T03:24:08.168185+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"165'1992","reported_seq":2856,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.769796+0000","last_change":"2024-09-22T03:24:07.048121+0000","last_active":"2024-09-22T03:24:08.769796+0000","last_peered":"2024-09-22T03:24:08.769796+0000","last_clean":"2024-09-22T03:24:08.769796+0000","last_became_active":"2024-09-22T03:24:02.365735+0000","last_became_peered":"2024-09-22T03:24:02.365735+0000","last_unstale":"2024-09-22T03:24:08.769796+0000","last_undegraded":"2024-09-22T03:24:08.769796+0000","last_fullsized":"2024-09-22T03:24:08.769796+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1992,"log_dups_size":0,"ondisk_log_size":1992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1993,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":356,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.128075+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:10.128075+0000","last_peered":"2024-09-22T03:24:10.128075+0000","last_clean":"2024-09-22T03:24:10.128075+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:10.128075+0000","last_undegraded":"2024-09-22T03:24:10.128075+0000","last_fullsized":"2024-09-22T03:24:10.128075+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":510,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.176675+0000","last_change":"2024-09-22T03:24:02.377618+0000","last_active":"2024-09-22T03:24:08.176675+0000","last_peered":"2024-09-22T03:24:08.176675+0000","last_clean":"2024-09-22T03:24:08.176675+0000","last_became_active":"2024-09-22T03:24:02.362178+0000","last_became_peered":"2024-09-22T03:24:02.362178+0000","last_unstale":"2024-09-22T03:24:08.176675+0000","last_undegraded":"2024-09-22T03:24:08.176675+0000","last_fullsized":"2024-09-22T03:24:08.176675+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"165'2128","reported_seq":3059,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.216833+0000","last_change":"2024-09-22T03:24:05.400340+0000","last_active":"2024-09-22T03:24:09.216833+0000","last_peered":"2024-09-22T03:24:09.216833+0000","last_clean":"2024-09-22T03:24:09.216833+0000","last_became_active":"2024-09-22T03:24:02.328698+0000","last_became_peered":"2024-09-22T03:24:02.328698+0000","last_unstale":"2024-09-22T03:24:09.216833+0000","last_undegraded":"2024-09-22T03:24:09.216833+0000","last_fullsized":"2024-09-22T03:24:09.216833+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2128,"log_dups_size":0,"ondisk_log_size":2128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2129,"num_write_kb":70720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":173,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.130663+0000","last_change":"2024-09-22T03:24:03.169310+0000","last_active":"2024-09-22T03:24:10.130663+0000","last_peered":"2024-09-22T03:24:10.130663+0000","last_clean":"2024-09-22T03:24:10.130663+0000","last_became_active":"2024-09-22T03:24:02.324336+0000","last_became_peered":"2024-09-22T03:24:02.324336+0000","last_unstale":"2024-09-22T03:24:10.130663+0000","last_undegraded":"2024-09-22T03:24:10.130663+0000","last_fullsized":"2024-09-22T03:24:10.130663+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"166'2192","reported_seq":2981,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:13.100095+0000","last_change":"2024-09-22T03:23:56.170283+0000","last_active":"2024-09-22T03:24:13.100095+0000","last_peered":"2024-09-22T03:24:13.100095+0000","last_clean":"2024-09-22T03:24:13.100095+0000","last_became_active":"2024-09-22T03:23:56.158439+0000","last_became_peered":"2024-09-22T03:23:56.158439+0000","last_unstale":"2024-09-22T03:24:13.100095+0000","last_undegraded":"2024-09-22T03:24:13.100095+0000","last_fullsized":"2024-09-22T03:24:13.100095+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2193,"num_write_kb":70464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":348,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.167517+0000","last_change":"2024-09-22T03:23:57.117265+0000","last_active":"2024-09-22T03:24:08.167517+0000","last_peered":"2024-09-22T03:24:08.167517+0000","last_clean":"2024-09-22T03:24:08.167517+0000","last_became_active":"2024-09-22T03:23:56.204767+0000","last_became_peered":"2024-09-22T03:23:56.204767+0000","last_unstale":"2024-09-22T03:24:08.167517+0000","last_undegraded":"2024-09-22T03:24:08.167517+0000","last_fullsized":"2024-09-22T03:24:08.167517+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"165'1880","reported_seq":2179,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:09.219945+0000","last_change":"2024-09-22T03:24:03.658213+0000","last_active":"2024-09-22T03:24:09.219945+0000","last_peered":"2024-09-22T03:24:09.219945+0000","last_clean":"2024-09-22T03:24:09.219945+0000","last_became_active":"2024-09-22T03:24:02.289524+0000","last_became_peered":"2024-09-22T03:24:02.289524+0000","last_unstale":"2024-09-22T03:24:09.219945+0000","last_undegraded":"2024-09-22T03:24:09.219945+0000","last_fullsized":"2024-09-22T03:24:09.219945+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":1880,"log_dups_size":0,"ondisk_log_size":1880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1881,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":291,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.129676+0000","last_change":"2024-09-22T03:24:03.155475+0000","last_active":"2024-09-22T03:24:10.129676+0000","last_peered":"2024-09-22T03:24:10.129676+0000","last_clean":"2024-09-22T03:24:10.129676+0000","last_became_active":"2024-09-22T03:24:02.266145+0000","last_became_peered":"2024-09-22T03:24:02.266145+0000","last_unstale":"2024-09-22T03:24:10.129676+0000","last_undegraded":"2024-09-22T03:24:10.129676+0000","last_fullsized":"2024-09-22T03:24:10.129676+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"165'2046","reported_seq":2797,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-22T03:24:08.936769+0000","last_change":"2024-09-22T03:24:04.680619+0000","last_active":"2024-09-22T03:24:08.936769+0000","last_peered":"2024-09-22T03:24:08.936769+0000","last_clean":"2024-09-22T03:24:08.936769+0000","last_became_active":"2024-09-22T03:24:02.316697+0000","last_became_peered":"2024-09-22T03:24:02.316697+0000","last_unstale":"2024-09-22T03:24:08.936769+0000","last_undegraded":"2024-09-22T03:24:08.936769+0000","last_fullsized":"2024-09-22T03:24:08.936769+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:19:41.394868+0000","last_clean_scrub_stamp":"2024-09-22T03:19:41.394868+0000","objects_scrubbed":0,"log_size":2046,"log_dups_size":0,"ondisk_log_size":2046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":246,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.130829+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:10.130829+0000","last_peered":"2024-09-22T03:24:10.130829+0000","last_clean":"2024-09-22T03:24:10.130829+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:10.130829+0000","last_undegraded":"2024-09-22T03:24:10.130829+0000","last_fullsized":"2024-09-22T03:24:10.130829+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":70189080,"num_objects":1072,"num_object_clones":0,"num_object_copies":3216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":32802,"num_write_kb":1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5876,"num_bytes_recovered":21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12132352,"data_stored":193921096,"data_compressed":9208408,"data_compressed_allocated":12120064,"data_compressed_original":193921024,"omap_allocated":0,"internal_metadata":0},"log_size":32786,"ondisk_log_size":32786,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65224,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":62720,"kb_avail":94306616,"statfs":{"total":96636764160,"available":96569974784,"internally_reserved":0,"allocated":2547712,"data_stored":19243648,"data_compressed":886920,"data_compressed_allocated":1167360,"data_compressed_original":18677760,"omap_allocated":0,"internal_metadata":64225280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":156,"seq":670014898190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51704,"kb_used_data":6052,"kb_used_omap":0,"kb_used_meta":45632,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":6197248,"data_stored":77371330,"data_compressed":3649228,"data_compressed_allocated":4804608,"data_compressed_original":76791808,"omap_allocated":0,"internal_metadata":46727168},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50752,"kb_used_data":3184,"kb_used_omap":0,"kb_used_meta":47552,"kb_avail":94321088,"statfs":{"total":96636764160,"available":96584794112,"internally_reserved":0,"allocated":3260416,"data_stored":30319752,"data_compressed":1414812,"data_compressed_allocated":1863680,"data_compressed_original":29736960,"omap_allocated":0,"internal_metadata":48693248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63844,"kb_used_data":7632,"kb_used_omap":0,"kb_used_meta":56192,"kb_avail":94307996,"statfs":{"total":96636764160,"available":96571387904,"internally_reserved":0,"allocated":7815168,"data_stored":103323567,"data_compressed":4881580,"data_compressed_allocated":6426624,"data_compressed_original":102744064,"omap_allocated":0,"internal_metadata":57540608},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5873664,"data_stored":93913112,"data_compressed":4459496,"data_compressed_allocated":5869568,"data_compressed_original":93913088,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1081344,"data_stored":17301504,"data_compressed":821568,"data_compressed_allocated":1081344,"data_compressed_original":17301504,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4767744,"data_stored":76218392,"data_compressed":3619256,"data_compressed_allocated":4763648,"data_compressed_original":76218368,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6488088,"data_compressed":308088,"data_compressed_allocated":405504,"data_compressed_original":6488064,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:13.879 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:13.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:13.979+0000 7fb4d3397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:13.983+0000 7fb4d3397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:16.054 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:16.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:16.167+0000 7f49ee997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:16.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:16.172+0000 7f49ee997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:17.983 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-22T03:24:17.983 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-22T03:24:18.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.097+0000 7fcf46197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.108+0000 7fcf46197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:18.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.353+0000 7fae65917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.358+0000 7fae65917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.449 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_2' removed 2024-09-22T03:24:18.466 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:24:18.466 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:24:18.466 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-22T03:24:18.467 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_3 16 2024-09-22T03:24:18.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.633+0000 7fb4ad46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.643+0000 7fb4ad46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.816 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 2, 1] 2024-09-22T03:24:18.816 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:24:18.816 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:18.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.931+0000 7fd3d2658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:18.939+0000 7fd3d2658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.138 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:19.139 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:19.151 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-09-22T03:24:18.449533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":220112,"kb_used_data":7952,"kb_used_omap":0,"kb_used_meta":212096,"kb_avail":377267248,"statfs":{"total":386547056640,"available":386321661952,"internally_reserved":0,"allocated":8142848,"data_stored":43225117,"data_compressed":1950892,"data_compressed_allocated":2572288,"data_compressed_original":40910848,"omap_allocated":0,"internal_metadata":217186304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-92930072,"num_objects":-1419,"num_object_clones":0,"num_object_copies":-4257,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-32455,"num_write_kb":-1083905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5876,"num_bytes_recovered":-21168152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.058289"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":274,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.125765+0000","last_change":"2024-09-22T03:23:57.118798+0000","last_active":"2024-09-22T03:24:10.125765+0000","last_peered":"2024-09-22T03:24:10.125765+0000","last_clean":"2024-09-22T03:24:10.125765+0000","last_became_active":"2024-09-22T03:23:56.167489+0000","last_became_peered":"2024-09-22T03:23:56.167489+0000","last_unstale":"2024-09-22T03:24:10.125765+0000","last_undegraded":"2024-09-22T03:24:10.125765+0000","last_fullsized":"2024-09-22T03:24:10.125765+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":394,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.143972+0000","last_change":"2024-09-22T03:24:03.145088+0000","last_active":"2024-09-22T03:24:10.143972+0000","last_peered":"2024-09-22T03:24:10.143972+0000","last_clean":"2024-09-22T03:24:10.143972+0000","last_became_active":"2024-09-22T03:24:02.330151+0000","last_became_peered":"2024-09-22T03:24:02.330151+0000","last_unstale":"2024-09-22T03:24:10.143972+0000","last_undegraded":"2024-09-22T03:24:10.143972+0000","last_fullsized":"2024-09-22T03:24:10.143972+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":442,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.144503+0000","last_change":"2024-09-22T03:24:03.155372+0000","last_active":"2024-09-22T03:24:10.144503+0000","last_peered":"2024-09-22T03:24:10.144503+0000","last_clean":"2024-09-22T03:24:10.144503+0000","last_became_active":"2024-09-22T03:24:02.225075+0000","last_became_peered":"2024-09-22T03:24:02.225075+0000","last_unstale":"2024-09-22T03:24:10.144503+0000","last_undegraded":"2024-09-22T03:24:10.144503+0000","last_fullsized":"2024-09-22T03:24:10.144503+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":356,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.128075+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:10.128075+0000","last_peered":"2024-09-22T03:24:10.128075+0000","last_clean":"2024-09-22T03:24:10.128075+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:10.128075+0000","last_undegraded":"2024-09-22T03:24:10.128075+0000","last_fullsized":"2024-09-22T03:24:10.128075+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":513,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.143065+0000","last_change":"2024-09-22T03:24:02.377618+0000","last_active":"2024-09-22T03:24:10.143065+0000","last_peered":"2024-09-22T03:24:10.143065+0000","last_clean":"2024-09-22T03:24:10.143065+0000","last_became_active":"2024-09-22T03:24:02.362178+0000","last_became_peered":"2024-09-22T03:24:02.362178+0000","last_unstale":"2024-09-22T03:24:10.143065+0000","last_undegraded":"2024-09-22T03:24:10.143065+0000","last_fullsized":"2024-09-22T03:24:10.143065+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":173,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.130663+0000","last_change":"2024-09-22T03:24:03.169310+0000","last_active":"2024-09-22T03:24:10.130663+0000","last_peered":"2024-09-22T03:24:10.130663+0000","last_clean":"2024-09-22T03:24:10.130663+0000","last_became_active":"2024-09-22T03:24:02.324336+0000","last_became_peered":"2024-09-22T03:24:02.324336+0000","last_unstale":"2024-09-22T03:24:10.130663+0000","last_undegraded":"2024-09-22T03:24:10.130663+0000","last_fullsized":"2024-09-22T03:24:10.130663+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":351,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.139964+0000","last_change":"2024-09-22T03:23:57.117265+0000","last_active":"2024-09-22T03:24:10.139964+0000","last_peered":"2024-09-22T03:24:10.139964+0000","last_clean":"2024-09-22T03:24:10.139964+0000","last_became_active":"2024-09-22T03:23:56.204767+0000","last_became_peered":"2024-09-22T03:23:56.204767+0000","last_unstale":"2024-09-22T03:24:10.139964+0000","last_undegraded":"2024-09-22T03:24:10.139964+0000","last_fullsized":"2024-09-22T03:24:10.139964+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":291,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.129676+0000","last_change":"2024-09-22T03:24:03.155475+0000","last_active":"2024-09-22T03:24:10.129676+0000","last_peered":"2024-09-22T03:24:10.129676+0000","last_clean":"2024-09-22T03:24:10.129676+0000","last_became_active":"2024-09-22T03:24:02.266145+0000","last_became_peered":"2024-09-22T03:24:02.266145+0000","last_unstale":"2024-09-22T03:24:10.129676+0000","last_undegraded":"2024-09-22T03:24:10.129676+0000","last_fullsized":"2024-09-22T03:24:10.129676+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":246,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-22T03:24:10.130829+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:10.130829+0000","last_peered":"2024-09-22T03:24:10.130829+0000","last_clean":"2024-09-22T03:24:10.130829+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:10.130829+0000","last_undegraded":"2024-09-22T03:24:10.130829+0000","last_fullsized":"2024-09-22T03:24:10.130829+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68240,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":66880,"kb_avail":94303600,"statfs":{"total":96636764160,"available":96566886400,"internally_reserved":0,"allocated":1376256,"data_stored":565864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68485120},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":156,"seq":670014898194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51744,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":49792,"kb_avail":94320096,"statfs":{"total":96636764160,"available":96583778304,"internally_reserved":0,"allocated":1982464,"data_stored":9807008,"data_compressed":440756,"data_compressed_allocated":581632,"data_compressed_original":9224192,"omap_allocated":0,"internal_metadata":50987008},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36492,"kb_used_data":1404,"kb_used_omap":0,"kb_used_meta":35072,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1437696,"data_stored":1156232,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35913728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63636,"kb_used_data":3268,"kb_used_omap":0,"kb_used_meta":60352,"kb_avail":94308204,"statfs":{"total":96636764160,"available":96571600896,"internally_reserved":0,"allocated":3346432,"data_stored":31696013,"data_compressed":1480164,"data_compressed_allocated":1949696,"data_compressed_original":31113216,"omap_allocated":0,"internal_metadata":61800448},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:19.452 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_3' created 2024-09-22T03:24:19.471 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-22T03:24:19.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:19.579+0000 7f935c22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:19.591+0000 7f935c22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:20.435 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:20.455 INFO:teuthology.orchestra.run.smithi146.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-22T03:24:20.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:20.588+0000 7f25c706c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:20.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:20.594+0000 7f25c706c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:21.478 DEBUG:teuthology.orchestra.run.smithi146:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-22T03:24:21.481 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:24:21.481 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:24:21.537 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:hints = 1 2024-09-22T03:24:21.537 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:24:21.537 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object prefix: benchmark_data_smithi146_48150 2024-09-22T03:24:21.537 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:24:21.537 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:24:22.537 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 1 16 426 410 25.6235 25.625 0.0333809 0.0381938 2024-09-22T03:24:22.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:22.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:22.806+0000 7fc4d18a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:22.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:22.817+0000 7fc4d18a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:23.537 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 2 16 833 817 25.5291 25.4375 0.00860919 0.0384129 2024-09-22T03:24:24.153 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 2, 1] 2024-09-22T03:24:24.153 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:24:24.153 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:24.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:24.299+0000 7f616806c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:24.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:24.313+0000 7f616806c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:24.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 3 16 1229 1213 25.2688 24.75 0.055054 0.0393514 2024-09-22T03:24:24.652 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:24.653 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:24.675 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":563,"stamp":"2024-09-22T03:24:23.394379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16056883,"num_objects":240,"num_object_clones":0,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":375,"num_write_kb":17000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":328,"ondisk_log_size":328,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":203548,"kb_used_data":7772,"kb_used_omap":0,"kb_used_meta":195456,"kb_avail":377283812,"statfs":{"total":386547056640,"available":386338623488,"internally_reserved":0,"allocated":7958528,"data_stored":39189758,"data_compressed":1757651,"data_compressed_allocated":2318336,"data_compressed_original":36847616,"omap_allocated":0,"internal_metadata":200146944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15466496,"num_objects":236,"num_object_clones":0,"num_object_copies":708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":15104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.944845"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"170'10","reported_seq":23,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:22.030822+0000","last_change":"2024-09-22T03:24:20.554581+0000","last_active":"2024-09-22T03:24:22.030822+0000","last_peered":"2024-09-22T03:24:22.030822+0000","last_clean":"2024-09-22T03:24:22.030822+0000","last_became_active":"2024-09-22T03:24:20.548105+0000","last_became_peered":"2024-09-22T03:24:20.548105+0000","last_unstale":"2024-09-22T03:24:22.030822+0000","last_undegraded":"2024-09-22T03:24:22.030822+0000","last_fullsized":"2024-09-22T03:24:22.030822+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"170'14","reported_seq":27,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:22.048512+0000","last_change":"2024-09-22T03:24:20.582250+0000","last_active":"2024-09-22T03:24:22.048512+0000","last_peered":"2024-09-22T03:24:22.048512+0000","last_clean":"2024-09-22T03:24:22.048512+0000","last_became_active":"2024-09-22T03:24:20.575484+0000","last_became_peered":"2024-09-22T03:24:20.575484+0000","last_unstale":"2024-09-22T03:24:22.048512+0000","last_undegraded":"2024-09-22T03:24:22.048512+0000","last_fullsized":"2024-09-22T03:24:22.048512+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"170'41","reported_seq":54,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:23.108858+0000","last_change":"2024-09-22T03:24:20.560260+0000","last_active":"2024-09-22T03:24:23.108858+0000","last_peered":"2024-09-22T03:24:23.108858+0000","last_clean":"2024-09-22T03:24:23.108858+0000","last_became_active":"2024-09-22T03:24:20.556791+0000","last_became_peered":"2024-09-22T03:24:20.556791+0000","last_unstale":"2024-09-22T03:24:23.108858+0000","last_undegraded":"2024-09-22T03:24:23.108858+0000","last_fullsized":"2024-09-22T03:24:23.108858+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"170'13","reported_seq":26,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:22.014942+0000","last_change":"2024-09-22T03:24:20.549408+0000","last_active":"2024-09-22T03:24:22.014942+0000","last_peered":"2024-09-22T03:24:22.014942+0000","last_clean":"2024-09-22T03:24:22.014942+0000","last_became_active":"2024-09-22T03:24:20.540836+0000","last_became_peered":"2024-09-22T03:24:20.540836+0000","last_unstale":"2024-09-22T03:24:22.014942+0000","last_undegraded":"2024-09-22T03:24:22.014942+0000","last_fullsized":"2024-09-22T03:24:22.014942+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"170'37","reported_seq":50,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:23.017192+0000","last_change":"2024-09-22T03:24:20.538419+0000","last_active":"2024-09-22T03:24:23.017192+0000","last_peered":"2024-09-22T03:24:23.017192+0000","last_clean":"2024-09-22T03:24:23.017192+0000","last_became_active":"2024-09-22T03:24:20.533630+0000","last_became_peered":"2024-09-22T03:24:20.533630+0000","last_unstale":"2024-09-22T03:24:23.017192+0000","last_undegraded":"2024-09-22T03:24:23.017192+0000","last_fullsized":"2024-09-22T03:24:23.017192+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":286,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.469481+0000","last_change":"2024-09-22T03:23:57.118798+0000","last_active":"2024-09-22T03:24:21.469481+0000","last_peered":"2024-09-22T03:24:21.469481+0000","last_clean":"2024-09-22T03:24:21.469481+0000","last_became_active":"2024-09-22T03:23:56.167489+0000","last_became_peered":"2024-09-22T03:23:56.167489+0000","last_unstale":"2024-09-22T03:24:21.469481+0000","last_undegraded":"2024-09-22T03:24:21.469481+0000","last_fullsized":"2024-09-22T03:24:21.469481+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"170'42","reported_seq":55,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:23.076907+0000","last_change":"2024-09-22T03:24:20.543343+0000","last_active":"2024-09-22T03:24:23.076907+0000","last_peered":"2024-09-22T03:24:23.076907+0000","last_clean":"2024-09-22T03:24:23.076907+0000","last_became_active":"2024-09-22T03:24:20.537036+0000","last_became_peered":"2024-09-22T03:24:20.537036+0000","last_unstale":"2024-09-22T03:24:23.076907+0000","last_undegraded":"2024-09-22T03:24:23.076907+0000","last_fullsized":"2024-09-22T03:24:23.076907+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":397,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-22T03:24:18.471528+0000","last_change":"2024-09-22T03:24:03.145088+0000","last_active":"2024-09-22T03:24:18.471528+0000","last_peered":"2024-09-22T03:24:18.471528+0000","last_clean":"2024-09-22T03:24:18.471528+0000","last_became_active":"2024-09-22T03:24:02.330151+0000","last_became_peered":"2024-09-22T03:24:02.330151+0000","last_unstale":"2024-09-22T03:24:18.471528+0000","last_undegraded":"2024-09-22T03:24:18.471528+0000","last_fullsized":"2024-09-22T03:24:18.471528+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":445,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-22T03:24:18.485317+0000","last_change":"2024-09-22T03:24:03.155372+0000","last_active":"2024-09-22T03:24:18.485317+0000","last_peered":"2024-09-22T03:24:18.485317+0000","last_clean":"2024-09-22T03:24:18.485317+0000","last_became_active":"2024-09-22T03:24:02.225075+0000","last_became_peered":"2024-09-22T03:24:02.225075+0000","last_unstale":"2024-09-22T03:24:18.485317+0000","last_undegraded":"2024-09-22T03:24:18.485317+0000","last_fullsized":"2024-09-22T03:24:18.485317+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"170'54","reported_seq":67,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:23.071360+0000","last_change":"2024-09-22T03:24:20.508982+0000","last_active":"2024-09-22T03:24:23.071360+0000","last_peered":"2024-09-22T03:24:23.071360+0000","last_clean":"2024-09-22T03:24:23.071360+0000","last_became_active":"2024-09-22T03:24:20.505162+0000","last_became_peered":"2024-09-22T03:24:20.505162+0000","last_unstale":"2024-09-22T03:24:23.071360+0000","last_undegraded":"2024-09-22T03:24:23.071360+0000","last_fullsized":"2024-09-22T03:24:23.071360+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":368,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.468559+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:21.468559+0000","last_peered":"2024-09-22T03:24:21.468559+0000","last_clean":"2024-09-22T03:24:21.468559+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:21.468559+0000","last_undegraded":"2024-09-22T03:24:21.468559+0000","last_fullsized":"2024-09-22T03:24:21.468559+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":516,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-22T03:24:18.470454+0000","last_change":"2024-09-22T03:24:02.377618+0000","last_active":"2024-09-22T03:24:18.470454+0000","last_peered":"2024-09-22T03:24:18.470454+0000","last_clean":"2024-09-22T03:24:18.470454+0000","last_became_active":"2024-09-22T03:24:02.362178+0000","last_became_peered":"2024-09-22T03:24:02.362178+0000","last_unstale":"2024-09-22T03:24:18.470454+0000","last_undegraded":"2024-09-22T03:24:18.470454+0000","last_fullsized":"2024-09-22T03:24:18.470454+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":185,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.466422+0000","last_change":"2024-09-22T03:24:03.169310+0000","last_active":"2024-09-22T03:24:21.466422+0000","last_peered":"2024-09-22T03:24:21.466422+0000","last_clean":"2024-09-22T03:24:21.466422+0000","last_became_active":"2024-09-22T03:24:02.324336+0000","last_became_peered":"2024-09-22T03:24:02.324336+0000","last_unstale":"2024-09-22T03:24:21.466422+0000","last_undegraded":"2024-09-22T03:24:21.466422+0000","last_fullsized":"2024-09-22T03:24:21.466422+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"170'17","reported_seq":30,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:22.083250+0000","last_change":"2024-09-22T03:24:20.545991+0000","last_active":"2024-09-22T03:24:22.083250+0000","last_peered":"2024-09-22T03:24:22.083250+0000","last_clean":"2024-09-22T03:24:22.083250+0000","last_became_active":"2024-09-22T03:24:20.541319+0000","last_became_peered":"2024-09-22T03:24:20.541319+0000","last_unstale":"2024-09-22T03:24:22.083250+0000","last_undegraded":"2024-09-22T03:24:22.083250+0000","last_fullsized":"2024-09-22T03:24:22.083250+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":354,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-22T03:24:18.480688+0000","last_change":"2024-09-22T03:23:57.117265+0000","last_active":"2024-09-22T03:24:18.480688+0000","last_peered":"2024-09-22T03:24:18.480688+0000","last_clean":"2024-09-22T03:24:18.480688+0000","last_became_active":"2024-09-22T03:23:56.204767+0000","last_became_peered":"2024-09-22T03:23:56.204767+0000","last_unstale":"2024-09-22T03:24:18.480688+0000","last_undegraded":"2024-09-22T03:24:18.480688+0000","last_fullsized":"2024-09-22T03:24:18.480688+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":303,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.468105+0000","last_change":"2024-09-22T03:24:03.155475+0000","last_active":"2024-09-22T03:24:21.468105+0000","last_peered":"2024-09-22T03:24:21.468105+0000","last_clean":"2024-09-22T03:24:21.468105+0000","last_became_active":"2024-09-22T03:24:02.266145+0000","last_became_peered":"2024-09-22T03:24:02.266145+0000","last_unstale":"2024-09-22T03:24:21.468105+0000","last_undegraded":"2024-09-22T03:24:21.468105+0000","last_fullsized":"2024-09-22T03:24:21.468105+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:24:19.449317+0000","last_change":"2024-09-22T03:24:19.449317+0000","last_active":"2024-09-22T03:24:19.449317+0000","last_peered":"2024-09-22T03:24:19.449317+0000","last_clean":"2024-09-22T03:24:19.449317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:24:19.449317+0000","last_undegraded":"2024-09-22T03:24:19.449317+0000","last_fullsized":"2024-09-22T03:24:19.449317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":258,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.467434+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:21.467434+0000","last_peered":"2024-09-22T03:24:21.467434+0000","last_clean":"2024-09-22T03:24:21.467434+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:21.467434+0000","last_undegraded":"2024-09-22T03:24:21.467434+0000","last_fullsized":"2024-09-22T03:24:21.467434+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":15466496,"num_objects":236,"num_object_clones":0,"num_object_copies":708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":15104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2678784,"data_stored":42860544,"data_compressed":2034422,"data_compressed_allocated":2678784,"data_compressed_original":42860544,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":24,"acting":24,"num_store_stats":2},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599032,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52504,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":50240,"kb_avail":94319336,"statfs":{"total":96636764160,"available":96583000064,"internally_reserved":0,"allocated":2301952,"data_stored":14801300,"data_compressed":675007,"data_compressed_allocated":888832,"data_compressed_original":14221312,"omap_allocated":0,"internal_metadata":51445760},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":156,"seq":670014898199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51212,"kb_used_data":1404,"kb_used_omap":0,"kb_used_meta":49792,"kb_avail":94320628,"statfs":{"total":96636764160,"available":96584323072,"internally_reserved":0,"allocated":1437696,"data_stored":1156232,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":50987008},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36520,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":35072,"kb_avail":94335320,"statfs":{"total":96636764160,"available":96599367680,"internally_reserved":0,"allocated":1466368,"data_stored":1166751,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35913728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611827992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63312,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":60352,"kb_avail":94308528,"statfs":{"total":96636764160,"available":96571932672,"internally_reserved":0,"allocated":2752512,"data_stored":22065475,"data_compressed":1022700,"data_compressed_allocated":1347584,"data_compressed_original":21479424,"omap_allocated":0,"internal_metadata":61800448},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":12845056,"data_compressed":609664,"data_compressed_allocated":802816,"data_compressed_original":12845056,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1875968,"data_stored":30015488,"data_compressed":1424758,"data_compressed_allocated":1875968,"data_compressed_original":30015488,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:25.012 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:25.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:25.168+0000 7fc00c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:25.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:25.179+0000 7fc00c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:25.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 4 16 1648 1632 25.4979 26.1875 0.0445881 0.0390399 2024-09-22T03:24:26.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 5 16 2078 2062 25.7729 26.875 0.0444403 0.0386586 2024-09-22T03:24:27.386 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:27.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 6 16 2507 2491 25.9458 26.8125 0.0419636 0.0384097 2024-09-22T03:24:27.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:27.546+0000 7fc75d517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:27.554+0000 7fc75d517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:28.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 7 16 2921 2905 25.9354 25.875 0.0411803 0.0384693 2024-09-22T03:24:29.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 8 16 3331 3315 25.8964 25.625 0.0477505 0.0385122 2024-09-22T03:24:29.678 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 2, 1] 2024-09-22T03:24:29.678 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:24:29.678 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 3, 2, 1] 2024-09-22T03:24:29.678 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:24:29.678 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:24:29.720 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:29.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:29.820+0000 7f990ec23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:29.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:29.824+0000 7f990ec23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:30.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 9 16 3401 3385 23.5049 4.375 0.0212768 0.0384544 2024-09-22T03:24:31.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 10 16 3401 3385 21.1543 0 - 0.0384544 2024-09-22T03:24:32.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 11 16 3401 3385 19.2312 0 - 0.0384544 2024-09-22T03:24:33.538 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 12 16 3401 3385 17.6285 0 - 0.0384544 2024-09-22T03:24:33.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:24:34.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 13 16 3401 3385 16.2724 0 - 0.0384544 2024-09-22T03:24:35.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 14 16 3401 3385 15.11 0 - 0.0384544 2024-09-22T03:24:35.779 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:24:36.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 15 16 3401 3385 14.1027 0 - 0.0384544 2024-09-22T03:24:37.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 16 16 3401 3385 13.2213 0 - 0.0384544 2024-09-22T03:24:38.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 17 16 3401 3385 12.4435 0 - 0.0384544 2024-09-22T03:24:39.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:24:39.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 18 16 3401 3385 11.7522 0 - 0.0384544 2024-09-22T03:24:40.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 19 16 3401 3385 11.1336 0 - 0.0384544 2024-09-22T03:24:40.780 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [0, 3, 1] 2024-09-22T03:24:40.780 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:24:40.781 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-22T03:24:40.781 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:24:40.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:24:40.784 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:24:40.784 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:24:40.895 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:24:40.899 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:40.899 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:24:41.539 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:24:41.539254+0000 min lat: 0.00860919 max lat: 0.0975447 avg lat: 0.0384544 2024-09-22T03:24:41.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:24:41.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 20 16 3401 3385 10.577 0 - 0.0384544 2024-09-22T03:24:42.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 21 16 3401 3385 10.0733 0 - 0.0384544 2024-09-22T03:24:43.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 22 16 3401 3385 9.61535 0 - 0.0384544 2024-09-22T03:24:44.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 23 16 3401 3385 9.1973 0 - 0.0384544 2024-09-22T03:24:45.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 24 16 3401 3385 8.81407 0 - 0.0384544 2024-09-22T03:24:45.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:24:45.998 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:24:46.000 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:46.001 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:24:46.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 25 16 3401 3385 8.46151 0 - 0.0384544 2024-09-22T03:24:47.139 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:24:47.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:24:47.214 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:24:47.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 26 16 3401 3385 8.13607 0 - 0.0384544 2024-09-22T03:24:48.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 27 16 3401 3385 7.83472 0 - 0.0384544 2024-09-22T03:24:49.215 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:49.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:49.352+0000 7f9f3b6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:49.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:49.359+0000 7f9f3b6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:49.540 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 28 16 3470 3454 7.70891 0.226974 0.0454782 0.129619 2024-09-22T03:24:50.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 29 16 3868 3852 8.30076 24.875 0.0379115 0.120378 2024-09-22T03:24:51.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [ 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 686, 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.112008+0000" 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.183 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.112018+0000" 2024-09-22T03:24:51.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.184 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.185 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.185 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.112723+0000" 2024-09-22T03:24:51.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.114199+0000" 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.114327+0000" 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.187 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.114962+0000" 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.115445+0000" 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.116079+0000" 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.191 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.192 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.116576+0000" 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.122890+0000" 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.194 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.150955+0000" 2024-09-22T03:24:51.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 688, 2024-09-22T03:24:51.196 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.120016+0000" 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.120028+0000" 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.121068+0000" 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.122588+0000" 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.123970+0000" 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.200 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.124847+0000" 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.201 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.127907+0000" 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.128534+0000" 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.204 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.128755+0000" 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.132236+0000" 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 690, 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.130553+0000" 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.208 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.130560+0000" 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.130716+0000" 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.131078+0000" 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.210 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.131359+0000" 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.132110+0000" 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.132458+0000" 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.213 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.133475+0000" 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.135453+0000" 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.217 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.140577+0000" 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 693, 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.145871+0000" 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.145881+0000" 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.220 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.146918+0000" 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.221 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.147387+0000" 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.148528+0000" 2024-09-22T03:24:51.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.223 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.149393+0000" 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.149627+0000" 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.150116+0000" 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.227 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.150912+0000", 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "blocker": { 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "op_type": "CommonPGPipeline::get_obc", 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.228 INFO:teuthology.orchestra.run.smithi148.stdout: "waiting": 0, 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "held_by_operation_id": 693 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.230 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 636, 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.940585+0000" 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.940596+0000" 2024-09-22T03:24:51.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.941466+0000" 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.233 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.941612+0000" 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.234 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.941783+0000" 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.980274+0000" 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.942425+0000" 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.236 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.943284+0000" 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.237 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.943778+0000" 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.239 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.944317+0000" 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.240 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.950139+0000" 2024-09-22T03:24:51.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.242 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.978394+0000" 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 638, 2024-09-22T03:24:51.243 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.943985+0000" 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.944023+0000" 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.944867+0000" 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.245 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.947330+0000" 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.947568+0000" 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.987667+0000" 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.948822+0000" 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.950212+0000" 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.249 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.951186+0000" 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.952669+0000" 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.954719+0000" 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.985847+0000" 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 639, 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.953767+0000" 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.953772+0000" 2024-09-22T03:24:51.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.954255+0000" 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.954946+0000" 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.955052+0000" 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.997524+0000" 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.957249+0000" 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.958409+0000" 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.958641+0000" 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.959488+0000" 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.263 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.961683+0000" 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.264 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.996997+0000" 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 648, 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.975201+0000" 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.975206+0000" 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.976599+0000" 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.976961+0000" 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.977170+0000" 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.011413+0000" 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.978317+0000" 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.979572+0000" 2024-09-22T03:24:51.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.980511+0000" 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.273 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.980957+0000" 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.983009+0000" 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.010395+0000" 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 645, 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.970197+0000" 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.970204+0000" 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.970397+0000" 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.970465+0000" 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.971129+0000" 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.014286+0000" 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.281 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.971373+0000" 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.971888+0000" 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.972060+0000" 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.972238+0000" 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.974879+0000" 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.013183+0000" 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 649, 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.981662+0000" 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.289 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.981668+0000" 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.981821+0000" 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.981929+0000" 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.982021+0000" 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.015796+0000" 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.982423+0000" 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.982600+0000" 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.982994+0000" 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.983528+0000" 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.986734+0000" 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.015293+0000" 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 650, 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.985479+0000" 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.985484+0000" 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.985814+0000" 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.986043+0000" 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.986098+0000" 2024-09-22T03:24:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.017483+0000" 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.986953+0000" 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.987930+0000" 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.989648+0000" 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.989984+0000" 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.992314+0000" 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.017007+0000" 2024-09-22T03:24:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 657, 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.001262+0000" 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.001271+0000" 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.001374+0000" 2024-09-22T03:24:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.001740+0000" 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.001836+0000" 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.048463+0000" 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.002261+0000" 2024-09-22T03:24:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.003124+0000" 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.003326+0000" 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.004502+0000" 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.006611+0000" 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.047423+0000" 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 663, 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.021995+0000" 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.022006+0000" 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.023287+0000" 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.024506+0000" 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.024676+0000" 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.055206+0000" 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.025339+0000" 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.025882+0000" 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.027783+0000" 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.028852+0000" 2024-09-22T03:24:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.032426+0000" 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.054462+0000" 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 661, 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.017853+0000" 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.017863+0000" 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.018369+0000" 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.018692+0000" 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.018856+0000" 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.058163+0000" 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.019392+0000" 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.019635+0000" 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.020032+0000" 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.020636+0000" 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.024580+0000" 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.057489+0000" 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 664, 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.030855+0000" 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.030869+0000" 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.032030+0000" 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.032236+0000" 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.033237+0000" 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.075512+0000" 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.034447+0000" 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.034977+0000" 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.035102+0000" 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.035430+0000" 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.037241+0000" 2024-09-22T03:24:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.073187+0000" 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 672, 2024-09-22T03:24:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059227+0000" 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059231+0000" 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059628+0000" 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059701+0000" 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059767+0000" 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.084814+0000" 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059953+0000" 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.060196+0000" 2024-09-22T03:24:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.060369+0000" 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.060712+0000" 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.063972+0000" 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.084228+0000" 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 670, 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.046405+0000" 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.046418+0000" 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.046612+0000" 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.047007+0000" 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.047377+0000" 2024-09-22T03:24:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.089172+0000" 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.047956+0000" 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.048343+0000" 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.048609+0000" 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.049305+0000" 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.051765+0000" 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.088296+0000" 2024-09-22T03:24:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 677, 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.071991+0000" 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.072004+0000" 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.072579+0000" 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.073342+0000" 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.075387+0000" 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.094681+0000" 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.076307+0000" 2024-09-22T03:24:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.076566+0000" 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.077496+0000" 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.077702+0000" 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.079750+0000" 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.094235+0000" 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 671, 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.057063+0000" 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.057075+0000" 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.057471+0000" 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.057772+0000" 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.058012+0000" 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.098642+0000" 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.058474+0000" 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.058753+0000" 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.058867+0000" 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059050+0000" 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.061464+0000" 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.098149+0000" 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 680, 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.085033+0000" 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.085040+0000" 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.085141+0000" 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.085798+0000" 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.085912+0000" 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.106049+0000" 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.086151+0000" 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.086907+0000" 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.087036+0000" 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.087216+0000" 2024-09-22T03:24:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.089899+0000" 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.105183+0000" 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 673, 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.066042+0000" 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.066051+0000" 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.067003+0000" 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.067225+0000" 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.067327+0000" 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.125195+0000" 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.068354+0000" 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.069030+0000" 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.069371+0000" 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.069812+0000" 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.072478+0000" 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.123999+0000" 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 685, 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.106536+0000" 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.106543+0000" 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.107053+0000" 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.107194+0000" 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.107701+0000" 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.147327+0000" 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.110368+0000" 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.111043+0000" 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.111411+0000" 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.111871+0000" 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.115901+0000" 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.144981+0000" 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 683, 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.102125+0000" 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.102140+0000" 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.102422+0000" 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.102580+0000" 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.102690+0000" 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.150479+0000" 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.103394+0000" 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.103682+0000" 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.104081+0000" 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.104475+0000" 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.107072+0000" 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.148467+0000" 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 691, 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.139122+0000" 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.139135+0000" 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.139971+0000" 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.140365+0000" 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.141936+0000" 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.142155+0000" 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.142280+0000" 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.142374+0000" 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.142528+0000" 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.143230+0000" 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 461, 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.278131+0000" 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.278136+0000" 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.278364+0000" 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.278701+0000" 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.280210+0000" 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.308494+0000" 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.280431+0000" 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.280551+0000" 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.280737+0000" 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.280940+0000" 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.281510+0000" 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.308121+0000" 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 465, 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.286304+0000" 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.286310+0000" 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.473 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.286972+0000" 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.287174+0000" 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.287619+0000" 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.316003+0000" 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.287833+0000" 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.287952+0000" 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.288051+0000" 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.288209+0000" 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.288965+0000" 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.315709+0000" 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 466, 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.295978+0000" 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.295983+0000" 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.296061+0000" 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.296270+0000" 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.296480+0000" 2024-09-22T03:24:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.329866+0000" 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.296834+0000" 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.296962+0000" 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.297062+0000" 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.297321+0000" 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.298014+0000" 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.329450+0000" 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 468, 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.297705+0000" 2024-09-22T03:24:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.297711+0000" 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.297866+0000" 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.297968+0000" 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.299606+0000" 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.338676+0000" 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.299824+0000" 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.299956+0000" 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.300057+0000" 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.300203+0000" 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.301275+0000" 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.338399+0000" 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 469, 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.303437+0000" 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.303443+0000" 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.303802+0000" 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.304308+0000" 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.304568+0000" 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.341067+0000" 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.304818+0000" 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.305119+0000" 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.305281+0000" 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.305570+0000" 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.306371+0000" 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.340811+0000" 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 507, 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.441009+0000" 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.441015+0000" 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.441509+0000" 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.442069+0000" 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.443938+0000" 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.493506+0000" 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.444192+0000" 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.444300+0000" 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.444395+0000" 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.444537+0000" 2024-09-22T03:24:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.445210+0000" 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.493248+0000" 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 533, 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.543647+0000" 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.543658+0000" 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.543788+0000" 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.544015+0000" 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.544891+0000" 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.571785+0000" 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.545115+0000" 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.545230+0000" 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.545331+0000" 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.545493+0000" 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.546174+0000" 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.571366+0000" 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 537, 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.557121+0000" 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.557127+0000" 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.557581+0000" 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.557771+0000" 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.558533+0000" 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.591322+0000" 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.558734+0000" 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.558831+0000" 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.558931+0000" 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.559133+0000" 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.560514+0000" 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.591072+0000" 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 539, 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.562746+0000" 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.562755+0000" 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.562892+0000" 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.563951+0000" 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.564148+0000" 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.602392+0000" 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.564387+0000" 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.564497+0000" 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.564599+0000" 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.564763+0000" 2024-09-22T03:24:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.565414+0000" 2024-09-22T03:24:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.602088+0000" 2024-09-22T03:24:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 544, 2024-09-22T03:24:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.581541+0000" 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.581547+0000" 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.581813+0000" 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.581950+0000" 2024-09-22T03:24:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.583473+0000" 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.632537+0000" 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.583693+0000" 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.583802+0000" 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.583908+0000" 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.584056+0000" 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.584724+0000" 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.632280+0000" 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 563, 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.662981+0000" 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.662990+0000" 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.663193+0000" 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.663415+0000" 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.664810+0000" 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.684030+0000" 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.665034+0000" 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.665146+0000" 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.665242+0000" 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.665384+0000" 2024-09-22T03:24:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.665981+0000" 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.683747+0000" 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 587, 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.742440+0000" 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.742447+0000" 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.742519+0000" 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.742585+0000" 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.742717+0000" 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.793341+0000" 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.743093+0000" 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.743221+0000" 2024-09-22T03:24:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.743314+0000" 2024-09-22T03:24:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.743469+0000" 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.744197+0000" 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.793082+0000" 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 595, 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.778301+0000" 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.778312+0000" 2024-09-22T03:24:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.778498+0000" 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.779181+0000" 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.781633+0000" 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.817443+0000" 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.782027+0000" 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.782221+0000" 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.782388+0000" 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.782631+0000" 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.783591+0000" 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.817015+0000" 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 601, 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.803492+0000" 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.803499+0000" 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.803871+0000" 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.803965+0000" 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.804783+0000" 2024-09-22T03:24:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.842784+0000" 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.805120+0000" 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.805286+0000" 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.805439+0000" 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.805674+0000" 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.806285+0000" 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.842396+0000" 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 608, 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.835526+0000" 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.835536+0000" 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.836227+0000" 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.836509+0000" 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.839279+0000" 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.865807+0000" 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.839519+0000" 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.839624+0000" 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.839726+0000" 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.839892+0000" 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.840669+0000" 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.865537+0000" 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 614, 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.854836+0000" 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.854850+0000" 2024-09-22T03:24:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.854952+0000" 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.855011+0000" 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.855921+0000" 2024-09-22T03:24:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.887473+0000" 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.856209+0000" 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.856333+0000" 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.856453+0000" 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.856655+0000" 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.857686+0000" 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.887173+0000" 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 618, 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.870689+0000" 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.870700+0000" 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.871194+0000" 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.872052+0000" 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.872673+0000" 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.893581+0000" 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.873069+0000" 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.646 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.873250+0000" 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.647 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.873436+0000" 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.648 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.873723+0000" 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.650 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.874507+0000" 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.893255+0000" 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 652, 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.992170+0000" 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.992176+0000" 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.992374+0000" 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.992461+0000" 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.994200+0000" 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.021369+0000" 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.994580+0000" 2024-09-22T03:24:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.994761+0000" 2024-09-22T03:24:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.994927+0000" 2024-09-22T03:24:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.995169+0000" 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:50.996228+0000" 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.021094+0000" 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 660, 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.015658+0000" 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.015663+0000" 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.015890+0000" 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.016106+0000" 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.016503+0000" 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.059331+0000" 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.016897+0000" 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.017084+0000" 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.017255+0000" 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.017506+0000" 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.018570+0000" 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:24:51.058886+0000" 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-22T03:24:51.674 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:24:51.674 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:51.675 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 30 16 4264 4248 8.84898 24.75 0.0552801 0.112917 2024-09-22T03:24:51.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:51.820+0000 7f822e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:51.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:51.827+0000 7f822e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 31 15 4675 4660 9.39409 25.75 0.0323174 0.106392 2024-09-22T03:24:53.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 32 16 5071 5055 9.87193 24.6875 0.0697517 0.10124 2024-09-22T03:24:54.030 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:54.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:54.185+0000 7f3f11b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:54.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:54.196+0000 7f3f108f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:54.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 33 16 5461 5445 10.3113 24.375 0.0203243 0.0969202 2024-09-22T03:24:55.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 34 16 5856 5840 10.7341 24.6875 0.01722 0.0930834 2024-09-22T03:24:56.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:56.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 35 16 6253 6237 11.1363 24.8125 0.0557439 0.0897226 2024-09-22T03:24:56.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:56.584+0000 7ff40ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:56.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:56.592+0000 7ff40ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:56.676 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:24:56.677 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:24:56.677 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:56.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:56.827+0000 7f8b8f117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:56.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:56.840+0000 7f8b8f117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:57.154 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:24:57.155 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:24:57.178 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2024-09-22T03:24:55.401251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":321716787,"num_objects":4904,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5039,"num_write_kb":315496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4984,"ondisk_log_size":4984,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":289608,"kb_used_data":60808,"kb_used_omap":0,"kb_used_meta":228736,"kb_avail":377197752,"statfs":{"total":386547056640,"available":386250498048,"internally_reserved":0,"allocated":62267392,"data_stored":905005365,"data_compressed":42864138,"data_compressed_allocated":56426496,"data_compressed_original":902578176,"omap_allocated":0,"internal_metadata":234225664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":103219200,"num_objects":1575,"num_object_clones":0,"num_object_copies":4725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1575,"num_write_kb":100800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.263708"},"pg_stats":[{"pgid":"6.f","version":"174'310","reported_seq":335,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:52.976747+0000","last_change":"2024-09-22T03:24:49.284049+0000","last_active":"2024-09-22T03:24:52.976747+0000","last_peered":"2024-09-22T03:24:52.976747+0000","last_clean":"2024-09-22T03:24:52.976747+0000","last_became_active":"2024-09-22T03:24:49.277059+0000","last_became_peered":"2024-09-22T03:24:49.277059+0000","last_unstale":"2024-09-22T03:24:52.976747+0000","last_undegraded":"2024-09-22T03:24:52.976747+0000","last_fullsized":"2024-09-22T03:24:52.976747+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":311,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"174'276","reported_seq":302,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:52.084080+0000","last_change":"2024-09-22T03:24:49.243863+0000","last_active":"2024-09-22T03:24:52.084080+0000","last_peered":"2024-09-22T03:24:52.084080+0000","last_clean":"2024-09-22T03:24:52.084080+0000","last_became_active":"2024-09-22T03:24:49.238475+0000","last_became_peered":"2024-09-22T03:24:49.238475+0000","last_unstale":"2024-09-22T03:24:52.084080+0000","last_undegraded":"2024-09-22T03:24:52.084080+0000","last_fullsized":"2024-09-22T03:24:52.084080+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":277,"num_object_clones":0,"num_object_copies":831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":277,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"174'290","reported_seq":315,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:52.080399+0000","last_change":"2024-09-22T03:24:20.582250+0000","last_active":"2024-09-22T03:24:52.080399+0000","last_peered":"2024-09-22T03:24:52.080399+0000","last_clean":"2024-09-22T03:24:52.080399+0000","last_became_active":"2024-09-22T03:24:20.575484+0000","last_became_peered":"2024-09-22T03:24:20.575484+0000","last_unstale":"2024-09-22T03:24:52.080399+0000","last_undegraded":"2024-09-22T03:24:52.080399+0000","last_fullsized":"2024-09-22T03:24:52.080399+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"174'319","reported_seq":354,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:53.132628+0000","last_change":"2024-09-22T03:24:49.377720+0000","last_active":"2024-09-22T03:24:53.132628+0000","last_peered":"2024-09-22T03:24:53.132628+0000","last_clean":"2024-09-22T03:24:53.132628+0000","last_became_active":"2024-09-22T03:24:49.227549+0000","last_became_peered":"2024-09-22T03:24:49.227549+0000","last_unstale":"2024-09-22T03:24:53.132628+0000","last_undegraded":"2024-09-22T03:24:53.132628+0000","last_fullsized":"2024-09-22T03:24:53.132628+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":320,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"174'293","reported_seq":319,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:52.010844+0000","last_change":"2024-09-22T03:24:49.219983+0000","last_active":"2024-09-22T03:24:52.010844+0000","last_peered":"2024-09-22T03:24:52.010844+0000","last_clean":"2024-09-22T03:24:52.010844+0000","last_became_active":"2024-09-22T03:24:49.214900+0000","last_became_peered":"2024-09-22T03:24:49.214900+0000","last_unstale":"2024-09-22T03:24:52.010844+0000","last_undegraded":"2024-09-22T03:24:52.010844+0000","last_fullsized":"2024-09-22T03:24:52.010844+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":294,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"174'300","reported_seq":325,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:53.079926+0000","last_change":"2024-09-22T03:24:49.253782+0000","last_active":"2024-09-22T03:24:53.079926+0000","last_peered":"2024-09-22T03:24:53.079926+0000","last_clean":"2024-09-22T03:24:53.079926+0000","last_became_active":"2024-09-22T03:24:49.245555+0000","last_became_peered":"2024-09-22T03:24:49.245555+0000","last_unstale":"2024-09-22T03:24:53.079926+0000","last_undegraded":"2024-09-22T03:24:53.079926+0000","last_fullsized":"2024-09-22T03:24:53.079926+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"174'315","reported_seq":352,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:53.029315+0000","last_change":"2024-09-22T03:24:49.402389+0000","last_active":"2024-09-22T03:24:53.029315+0000","last_peered":"2024-09-22T03:24:53.029315+0000","last_clean":"2024-09-22T03:24:53.029315+0000","last_became_active":"2024-09-22T03:24:49.215003+0000","last_became_peered":"2024-09-22T03:24:49.215003+0000","last_unstale":"2024-09-22T03:24:53.029315+0000","last_undegraded":"2024-09-22T03:24:53.029315+0000","last_fullsized":"2024-09-22T03:24:53.029315+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709376,"num_objects":316,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":316,"num_write_kb":20224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"174'307","reported_seq":338,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:54.096661+0000","last_change":"2024-09-22T03:24:49.337699+0000","last_active":"2024-09-22T03:24:54.096661+0000","last_peered":"2024-09-22T03:24:54.096661+0000","last_clean":"2024-09-22T03:24:54.096661+0000","last_became_active":"2024-09-22T03:24:49.238244+0000","last_became_peered":"2024-09-22T03:24:49.238244+0000","last_unstale":"2024-09-22T03:24:54.096661+0000","last_undegraded":"2024-09-22T03:24:54.096661+0000","last_fullsized":"2024-09-22T03:24:54.096661+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":308,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"174'308","reported_seq":333,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:54.105626+0000","last_change":"2024-09-22T03:24:20.529927+0000","last_active":"2024-09-22T03:24:54.105626+0000","last_peered":"2024-09-22T03:24:54.105626+0000","last_clean":"2024-09-22T03:24:54.105626+0000","last_became_active":"2024-09-22T03:24:20.524882+0000","last_became_peered":"2024-09-22T03:24:20.524882+0000","last_unstale":"2024-09-22T03:24:54.105626+0000","last_undegraded":"2024-09-22T03:24:54.105626+0000","last_fullsized":"2024-09-22T03:24:54.105626+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":19776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":300,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131545+0000","last_change":"2024-09-22T03:24:50.133990+0000","last_active":"2024-09-22T03:24:50.131545+0000","last_peered":"2024-09-22T03:24:50.131545+0000","last_clean":"2024-09-22T03:24:50.131545+0000","last_became_active":"2024-09-22T03:24:49.230660+0000","last_became_peered":"2024-09-22T03:24:49.230660+0000","last_unstale":"2024-09-22T03:24:50.131545+0000","last_undegraded":"2024-09-22T03:24:50.131545+0000","last_fullsized":"2024-09-22T03:24:50.131545+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"174'303","reported_seq":328,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:53.114594+0000","last_change":"2024-09-22T03:24:20.543343+0000","last_active":"2024-09-22T03:24:53.114594+0000","last_peered":"2024-09-22T03:24:53.114594+0000","last_clean":"2024-09-22T03:24:53.114594+0000","last_became_active":"2024-09-22T03:24:20.537036+0000","last_became_peered":"2024-09-22T03:24:20.537036+0000","last_unstale":"2024-09-22T03:24:53.114594+0000","last_undegraded":"2024-09-22T03:24:53.114594+0000","last_fullsized":"2024-09-22T03:24:53.114594+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19922944,"num_objects":304,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":420,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131531+0000","last_change":"2024-09-22T03:24:50.133761+0000","last_active":"2024-09-22T03:24:50.131531+0000","last_peered":"2024-09-22T03:24:50.131531+0000","last_clean":"2024-09-22T03:24:50.131531+0000","last_became_active":"2024-09-22T03:24:49.245678+0000","last_became_peered":"2024-09-22T03:24:49.245678+0000","last_unstale":"2024-09-22T03:24:50.131531+0000","last_undegraded":"2024-09-22T03:24:50.131531+0000","last_fullsized":"2024-09-22T03:24:50.131531+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"174'291","reported_seq":322,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:52.960416+0000","last_change":"2024-09-22T03:24:49.407992+0000","last_active":"2024-09-22T03:24:52.960416+0000","last_peered":"2024-09-22T03:24:52.960416+0000","last_clean":"2024-09-22T03:24:52.960416+0000","last_became_active":"2024-09-22T03:24:49.262959+0000","last_became_peered":"2024-09-22T03:24:49.262959+0000","last_unstale":"2024-09-22T03:24:52.960416+0000","last_undegraded":"2024-09-22T03:24:52.960416+0000","last_fullsized":"2024-09-22T03:24:52.960416+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":292,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":436,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.135967+0000","last_change":"2024-09-22T03:24:50.139730+0000","last_active":"2024-09-22T03:24:50.135967+0000","last_peered":"2024-09-22T03:24:50.135967+0000","last_clean":"2024-09-22T03:24:50.135967+0000","last_became_active":"2024-09-22T03:24:49.263866+0000","last_became_peered":"2024-09-22T03:24:49.263866+0000","last_unstale":"2024-09-22T03:24:50.135967+0000","last_undegraded":"2024-09-22T03:24:50.135967+0000","last_fullsized":"2024-09-22T03:24:50.135967+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"174'324","reported_seq":349,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:53.039742+0000","last_change":"2024-09-22T03:24:20.508982+0000","last_active":"2024-09-22T03:24:53.039742+0000","last_peered":"2024-09-22T03:24:53.039742+0000","last_clean":"2024-09-22T03:24:53.039742+0000","last_became_active":"2024-09-22T03:24:20.505162+0000","last_became_peered":"2024-09-22T03:24:20.505162+0000","last_unstale":"2024-09-22T03:24:53.039742+0000","last_undegraded":"2024-09-22T03:24:53.039742+0000","last_fullsized":"2024-09-22T03:24:53.039742+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21299200,"num_objects":325,"num_object_clones":0,"num_object_copies":975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":325,"num_write_kb":20800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":380,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.130593+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:50.130593+0000","last_peered":"2024-09-22T03:24:50.130593+0000","last_clean":"2024-09-22T03:24:50.130593+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:50.130593+0000","last_undegraded":"2024-09-22T03:24:50.130593+0000","last_fullsized":"2024-09-22T03:24:50.130593+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":538,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131988+0000","last_change":"2024-09-22T03:24:49.229480+0000","last_active":"2024-09-22T03:24:50.131988+0000","last_peered":"2024-09-22T03:24:50.131988+0000","last_clean":"2024-09-22T03:24:50.131988+0000","last_became_active":"2024-09-22T03:24:49.225634+0000","last_became_peered":"2024-09-22T03:24:49.225634+0000","last_unstale":"2024-09-22T03:24:50.131988+0000","last_undegraded":"2024-09-22T03:24:50.131988+0000","last_fullsized":"2024-09-22T03:24:50.131988+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"174'330","reported_seq":355,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:54.201820+0000","last_change":"2024-09-22T03:24:20.555078+0000","last_active":"2024-09-22T03:24:54.201820+0000","last_peered":"2024-09-22T03:24:54.201820+0000","last_clean":"2024-09-22T03:24:54.201820+0000","last_became_active":"2024-09-22T03:24:20.549566+0000","last_became_peered":"2024-09-22T03:24:20.549566+0000","last_unstale":"2024-09-22T03:24:54.201820+0000","last_undegraded":"2024-09-22T03:24:54.201820+0000","last_fullsized":"2024-09-22T03:24:54.201820+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":331,"num_object_clones":0,"num_object_copies":993,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":21184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.125507+0000","last_change":"2024-09-22T03:24:50.129335+0000","last_active":"2024-09-22T03:24:50.125507+0000","last_peered":"2024-09-22T03:24:50.125507+0000","last_clean":"2024-09-22T03:24:50.125507+0000","last_became_active":"2024-09-22T03:24:49.229324+0000","last_became_peered":"2024-09-22T03:24:49.229324+0000","last_unstale":"2024-09-22T03:24:50.125507+0000","last_undegraded":"2024-09-22T03:24:50.125507+0000","last_fullsized":"2024-09-22T03:24:50.125507+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"174'299","reported_seq":325,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:52.082665+0000","last_change":"2024-09-22T03:24:49.226520+0000","last_active":"2024-09-22T03:24:52.082665+0000","last_peered":"2024-09-22T03:24:52.082665+0000","last_clean":"2024-09-22T03:24:52.082665+0000","last_became_active":"2024-09-22T03:24:49.222276+0000","last_became_peered":"2024-09-22T03:24:49.222276+0000","last_unstale":"2024-09-22T03:24:52.082665+0000","last_undegraded":"2024-09-22T03:24:52.082665+0000","last_fullsized":"2024-09-22T03:24:52.082665+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19660800,"num_objects":300,"num_object_clones":0,"num_object_copies":900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":300,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":336,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.132373+0000","last_change":"2024-09-22T03:24:50.146187+0000","last_active":"2024-09-22T03:24:50.132373+0000","last_peered":"2024-09-22T03:24:50.132373+0000","last_clean":"2024-09-22T03:24:50.132373+0000","last_became_active":"2024-09-22T03:24:49.269404+0000","last_became_peered":"2024-09-22T03:24:49.269404+0000","last_unstale":"2024-09-22T03:24:50.132373+0000","last_undegraded":"2024-09-22T03:24:50.132373+0000","last_fullsized":"2024-09-22T03:24:50.132373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"174'313","reported_seq":338,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:53.093856+0000","last_change":"2024-09-22T03:24:49.265910+0000","last_active":"2024-09-22T03:24:53.093856+0000","last_peered":"2024-09-22T03:24:53.093856+0000","last_clean":"2024-09-22T03:24:53.093856+0000","last_became_active":"2024-09-22T03:24:49.260555+0000","last_became_peered":"2024-09-22T03:24:49.260555+0000","last_unstale":"2024-09-22T03:24:53.093856+0000","last_undegraded":"2024-09-22T03:24:53.093856+0000","last_fullsized":"2024-09-22T03:24:53.093856+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":314,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":317,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.127634+0000","last_change":"2024-09-22T03:24:50.130812+0000","last_active":"2024-09-22T03:24:50.127634+0000","last_peered":"2024-09-22T03:24:50.127634+0000","last_clean":"2024-09-22T03:24:50.127634+0000","last_became_active":"2024-09-22T03:24:49.246771+0000","last_became_peered":"2024-09-22T03:24:49.246771+0000","last_unstale":"2024-09-22T03:24:50.127634+0000","last_undegraded":"2024-09-22T03:24:50.127634+0000","last_fullsized":"2024-09-22T03:24:50.127634+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"174'306","reported_seq":331,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:53.062199+0000","last_change":"2024-09-22T03:24:49.258029+0000","last_active":"2024-09-22T03:24:53.062199+0000","last_peered":"2024-09-22T03:24:53.062199+0000","last_clean":"2024-09-22T03:24:53.062199+0000","last_became_active":"2024-09-22T03:24:49.251533+0000","last_became_peered":"2024-09-22T03:24:49.251533+0000","last_unstale":"2024-09-22T03:24:53.062199+0000","last_undegraded":"2024-09-22T03:24:53.062199+0000","last_fullsized":"2024-09-22T03:24:53.062199+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":307,"num_object_clones":0,"num_object_copies":921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":307,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":270,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.123717+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:50.123717+0000","last_peered":"2024-09-22T03:24:50.123717+0000","last_clean":"2024-09-22T03:24:50.123717+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:50.123717+0000","last_undegraded":"2024-09-22T03:24:50.123717+0000","last_fullsized":"2024-09-22T03:24:50.123717+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":321126400,"num_objects":4900,"num_object_clones":0,"num_object_copies":14700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4900,"num_write_kb":313600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":60379136,"data_stored":966066176,"data_compressed":45870662,"data_compressed_allocated":60379136,"data_compressed_original":966066176,"omap_allocated":0,"internal_metadata":0},"log_size":4884,"ondisk_log_size":4884,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599062,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76656,"kb_used_data":13920,"kb_used_omap":0,"kb_used_meta":62720,"kb_avail":94295184,"statfs":{"total":96636764160,"available":96558268416,"internally_reserved":0,"allocated":14254080,"data_stored":205525178,"data_compressed":9730468,"data_compressed_allocated":12808192,"data_compressed_original":204931072,"omap_allocated":0,"internal_metadata":64225280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":172,"seq":738734374916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54000,"kb_used_data":13920,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94317840,"statfs":{"total":96636764160,"available":96581468160,"internally_reserved":0,"allocated":14254080,"data_stored":205132506,"data_compressed":9713769,"data_compressed_allocated":12787712,"data_compressed_original":204521472,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62544,"kb_used_data":14976,"kb_used_omap":0,"kb_used_meta":47552,"kb_avail":94309296,"statfs":{"total":96636764160,"available":96572719104,"internally_reserved":0,"allocated":15335424,"data_stored":222434010,"data_compressed":10535166,"data_compressed_allocated":13869056,"data_compressed_original":221822976,"omap_allocated":0,"internal_metadata":48693248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611828027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96408,"kb_used_data":17992,"kb_used_omap":0,"kb_used_meta":78400,"kb_avail":94275432,"statfs":{"total":96636764160,"available":96538042368,"internally_reserved":0,"allocated":18423808,"data_stored":271913671,"data_compressed":12884735,"data_compressed_allocated":16961536,"data_compressed_original":271302656,"omap_allocated":0,"internal_metadata":80281600},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17698816,"data_stored":283181056,"data_compressed":13446043,"data_compressed_allocated":17698816,"data_compressed_original":283181056,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15065088,"data_stored":241041408,"data_compressed":11445018,"data_compressed_allocated":15065088,"data_compressed_original":241041408,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13901824,"data_stored":222429184,"data_compressed":10561381,"data_compressed_allocated":13901824,"data_compressed_original":222429184,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13713408,"data_stored":219414528,"data_compressed":10418220,"data_compressed_allocated":13713408,"data_compressed_original":219414528,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:57.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 36 16 6621 6605 11.4658 23 0.0770386 0.0871214 2024-09-22T03:24:58.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 37 15 7000 6985 11.7977 23.75 0.0543929 0.0847055 2024-09-22T03:24:58.784 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:58.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:58.936+0000 7f6e6c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:58.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:24:58.945+0000 7f6e6c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:59.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 38 16 7361 7345 12.0793 22.5 0.0581656 0.0827233 2024-09-22T03:25:00.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 39 16 7760 7744 12.4089 24.9375 0.055321 0.0805421 2024-09-22T03:25:01.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:01.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:01.312+0000 7fcc66b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:01.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:01.319+0000 7fcc66b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:01.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:25:01.541297+0000 min lat: 0.00860919 max lat: 19.7689 avg lat: 0.0788601 2024-09-22T03:25:01.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:25:01.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 40 15 8126 8111 12.6721 22.9375 0.0587618 0.0788601 2024-09-22T03:25:02.179 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:25:02.180 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:02.180 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:02.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:02.330+0000 7f1da9d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:02.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:02.341+0000 7f1da9d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:02.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 41 16 8510 8494 12.9468 23.9375 0.0437021 0.0771971 2024-09-22T03:25:02.610 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:25:02.610 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:25:02.629 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2024-09-22T03:25:01.402254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":447480371,"num_objects":6823,"num_object_clones":0,"num_object_copies":20469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6958,"num_write_kb":438312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6903,"ondisk_log_size":6903,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":300656,"kb_used_data":84336,"kb_used_omap":0,"kb_used_meta":216256,"kb_avail":377186704,"statfs":{"total":386547056640,"available":386239184896,"internally_reserved":0,"allocated":86360064,"data_stored":1290488117,"data_compressed":61168922,"data_compressed_allocated":80519168,"data_compressed_original":1288060928,"omap_allocated":0,"internal_metadata":221446144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":227344384,"num_objects":3469,"num_object_clones":0,"num_object_copies":10407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3469,"num_write_kb":222016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.291364"},"pg_stats":[{"pgid":"6.f","version":"174'430","reported_seq":455,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:58.095035+0000","last_change":"2024-09-22T03:24:49.284049+0000","last_active":"2024-09-22T03:24:58.095035+0000","last_peered":"2024-09-22T03:24:58.095035+0000","last_clean":"2024-09-22T03:24:58.095035+0000","last_became_active":"2024-09-22T03:24:49.277059+0000","last_became_peered":"2024-09-22T03:24:49.277059+0000","last_unstale":"2024-09-22T03:24:58.095035+0000","last_undegraded":"2024-09-22T03:24:58.095035+0000","last_fullsized":"2024-09-22T03:24:58.095035+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28246016,"num_objects":431,"num_object_clones":0,"num_object_copies":1293,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":431,"num_write_kb":27584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"174'410","reported_seq":436,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:57.000306+0000","last_change":"2024-09-22T03:24:49.243863+0000","last_active":"2024-09-22T03:24:57.000306+0000","last_peered":"2024-09-22T03:24:57.000306+0000","last_clean":"2024-09-22T03:24:57.000306+0000","last_became_active":"2024-09-22T03:24:49.238475+0000","last_became_peered":"2024-09-22T03:24:49.238475+0000","last_unstale":"2024-09-22T03:24:57.000306+0000","last_undegraded":"2024-09-22T03:24:57.000306+0000","last_fullsized":"2024-09-22T03:24:57.000306+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":411,"num_object_clones":0,"num_object_copies":1233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":411,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"174'390","reported_seq":415,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:57.040973+0000","last_change":"2024-09-22T03:24:20.582250+0000","last_active":"2024-09-22T03:24:57.040973+0000","last_peered":"2024-09-22T03:24:57.040973+0000","last_clean":"2024-09-22T03:24:57.040973+0000","last_became_active":"2024-09-22T03:24:20.575484+0000","last_became_peered":"2024-09-22T03:24:20.575484+0000","last_unstale":"2024-09-22T03:24:57.040973+0000","last_undegraded":"2024-09-22T03:24:57.040973+0000","last_fullsized":"2024-09-22T03:24:57.040973+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":391,"num_object_clones":0,"num_object_copies":1173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":391,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"174'447","reported_seq":482,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:58.059397+0000","last_change":"2024-09-22T03:24:49.377720+0000","last_active":"2024-09-22T03:24:58.059397+0000","last_peered":"2024-09-22T03:24:58.059397+0000","last_clean":"2024-09-22T03:24:58.059397+0000","last_became_active":"2024-09-22T03:24:49.227549+0000","last_became_peered":"2024-09-22T03:24:49.227549+0000","last_unstale":"2024-09-22T03:24:58.059397+0000","last_undegraded":"2024-09-22T03:24:58.059397+0000","last_fullsized":"2024-09-22T03:24:58.059397+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":448,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"174'421","reported_seq":447,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:57.087889+0000","last_change":"2024-09-22T03:24:49.219983+0000","last_active":"2024-09-22T03:24:57.087889+0000","last_peered":"2024-09-22T03:24:57.087889+0000","last_clean":"2024-09-22T03:24:57.087889+0000","last_became_active":"2024-09-22T03:24:49.214900+0000","last_became_peered":"2024-09-22T03:24:49.214900+0000","last_unstale":"2024-09-22T03:24:57.087889+0000","last_undegraded":"2024-09-22T03:24:57.087889+0000","last_fullsized":"2024-09-22T03:24:57.087889+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27656192,"num_objects":422,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":422,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"174'407","reported_seq":432,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:57.929680+0000","last_change":"2024-09-22T03:24:49.253782+0000","last_active":"2024-09-22T03:24:57.929680+0000","last_peered":"2024-09-22T03:24:57.929680+0000","last_clean":"2024-09-22T03:24:57.929680+0000","last_became_active":"2024-09-22T03:24:49.245555+0000","last_became_peered":"2024-09-22T03:24:49.245555+0000","last_unstale":"2024-09-22T03:24:57.929680+0000","last_undegraded":"2024-09-22T03:24:57.929680+0000","last_fullsized":"2024-09-22T03:24:57.929680+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26738688,"num_objects":408,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":26112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"174'436","reported_seq":473,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:58.069565+0000","last_change":"2024-09-22T03:24:49.402389+0000","last_active":"2024-09-22T03:24:58.069565+0000","last_peered":"2024-09-22T03:24:58.069565+0000","last_clean":"2024-09-22T03:24:58.069565+0000","last_became_active":"2024-09-22T03:24:49.215003+0000","last_became_peered":"2024-09-22T03:24:49.215003+0000","last_unstale":"2024-09-22T03:24:58.069565+0000","last_undegraded":"2024-09-22T03:24:58.069565+0000","last_fullsized":"2024-09-22T03:24:58.069565+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28639232,"num_objects":437,"num_object_clones":0,"num_object_copies":1311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":437,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"174'434","reported_seq":465,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:59.171578+0000","last_change":"2024-09-22T03:24:49.337699+0000","last_active":"2024-09-22T03:24:59.171578+0000","last_peered":"2024-09-22T03:24:59.171578+0000","last_clean":"2024-09-22T03:24:59.171578+0000","last_became_active":"2024-09-22T03:24:49.238244+0000","last_became_peered":"2024-09-22T03:24:49.238244+0000","last_unstale":"2024-09-22T03:24:59.171578+0000","last_undegraded":"2024-09-22T03:24:59.171578+0000","last_fullsized":"2024-09-22T03:24:59.171578+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28508160,"num_objects":435,"num_object_clones":0,"num_object_copies":1305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":435,"num_write_kb":27840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"174'421","reported_seq":446,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:59.189276+0000","last_change":"2024-09-22T03:24:20.529927+0000","last_active":"2024-09-22T03:24:59.189276+0000","last_peered":"2024-09-22T03:24:59.189276+0000","last_clean":"2024-09-22T03:24:59.189276+0000","last_became_active":"2024-09-22T03:24:20.524882+0000","last_became_peered":"2024-09-22T03:24:20.524882+0000","last_unstale":"2024-09-22T03:24:59.189276+0000","last_undegraded":"2024-09-22T03:24:59.189276+0000","last_fullsized":"2024-09-22T03:24:59.189276+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27656192,"num_objects":422,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":422,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":300,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131545+0000","last_change":"2024-09-22T03:24:50.133990+0000","last_active":"2024-09-22T03:24:50.131545+0000","last_peered":"2024-09-22T03:24:50.131545+0000","last_clean":"2024-09-22T03:24:50.131545+0000","last_became_active":"2024-09-22T03:24:49.230660+0000","last_became_peered":"2024-09-22T03:24:49.230660+0000","last_unstale":"2024-09-22T03:24:50.131545+0000","last_undegraded":"2024-09-22T03:24:50.131545+0000","last_fullsized":"2024-09-22T03:24:50.131545+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"174'415","reported_seq":440,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:58.003863+0000","last_change":"2024-09-22T03:24:20.543343+0000","last_active":"2024-09-22T03:24:58.003863+0000","last_peered":"2024-09-22T03:24:58.003863+0000","last_clean":"2024-09-22T03:24:58.003863+0000","last_became_active":"2024-09-22T03:24:20.537036+0000","last_became_peered":"2024-09-22T03:24:20.537036+0000","last_unstale":"2024-09-22T03:24:58.003863+0000","last_undegraded":"2024-09-22T03:24:58.003863+0000","last_fullsized":"2024-09-22T03:24:58.003863+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27262976,"num_objects":416,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":26624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":420,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131531+0000","last_change":"2024-09-22T03:24:50.133761+0000","last_active":"2024-09-22T03:24:50.131531+0000","last_peered":"2024-09-22T03:24:50.131531+0000","last_clean":"2024-09-22T03:24:50.131531+0000","last_became_active":"2024-09-22T03:24:49.245678+0000","last_became_peered":"2024-09-22T03:24:49.245678+0000","last_unstale":"2024-09-22T03:24:50.131531+0000","last_undegraded":"2024-09-22T03:24:50.131531+0000","last_fullsized":"2024-09-22T03:24:50.131531+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"174'414","reported_seq":445,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:58.082127+0000","last_change":"2024-09-22T03:24:49.407992+0000","last_active":"2024-09-22T03:24:58.082127+0000","last_peered":"2024-09-22T03:24:58.082127+0000","last_clean":"2024-09-22T03:24:58.082127+0000","last_became_active":"2024-09-22T03:24:49.262959+0000","last_became_peered":"2024-09-22T03:24:49.262959+0000","last_unstale":"2024-09-22T03:24:58.082127+0000","last_undegraded":"2024-09-22T03:24:58.082127+0000","last_fullsized":"2024-09-22T03:24:58.082127+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":26560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":436,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.135967+0000","last_change":"2024-09-22T03:24:50.139730+0000","last_active":"2024-09-22T03:24:50.135967+0000","last_peered":"2024-09-22T03:24:50.135967+0000","last_clean":"2024-09-22T03:24:50.135967+0000","last_became_active":"2024-09-22T03:24:49.263866+0000","last_became_peered":"2024-09-22T03:24:49.263866+0000","last_unstale":"2024-09-22T03:24:50.135967+0000","last_undegraded":"2024-09-22T03:24:50.135967+0000","last_fullsized":"2024-09-22T03:24:50.135967+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"174'445","reported_seq":470,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:57.917262+0000","last_change":"2024-09-22T03:24:20.508982+0000","last_active":"2024-09-22T03:24:57.917262+0000","last_peered":"2024-09-22T03:24:57.917262+0000","last_clean":"2024-09-22T03:24:57.917262+0000","last_became_active":"2024-09-22T03:24:20.505162+0000","last_became_peered":"2024-09-22T03:24:20.505162+0000","last_unstale":"2024-09-22T03:24:57.917262+0000","last_undegraded":"2024-09-22T03:24:57.917262+0000","last_fullsized":"2024-09-22T03:24:57.917262+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29229056,"num_objects":446,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":446,"num_write_kb":28544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":380,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.130593+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:50.130593+0000","last_peered":"2024-09-22T03:24:50.130593+0000","last_clean":"2024-09-22T03:24:50.130593+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:50.130593+0000","last_undegraded":"2024-09-22T03:24:50.130593+0000","last_fullsized":"2024-09-22T03:24:50.130593+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":538,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131988+0000","last_change":"2024-09-22T03:24:49.229480+0000","last_active":"2024-09-22T03:24:50.131988+0000","last_peered":"2024-09-22T03:24:50.131988+0000","last_clean":"2024-09-22T03:24:50.131988+0000","last_became_active":"2024-09-22T03:24:49.225634+0000","last_became_peered":"2024-09-22T03:24:49.225634+0000","last_unstale":"2024-09-22T03:24:50.131988+0000","last_undegraded":"2024-09-22T03:24:50.131988+0000","last_fullsized":"2024-09-22T03:24:50.131988+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"174'448","reported_seq":473,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:59.127722+0000","last_change":"2024-09-22T03:24:20.555078+0000","last_active":"2024-09-22T03:24:59.127722+0000","last_peered":"2024-09-22T03:24:59.127722+0000","last_clean":"2024-09-22T03:24:59.127722+0000","last_became_active":"2024-09-22T03:24:20.549566+0000","last_became_peered":"2024-09-22T03:24:20.549566+0000","last_unstale":"2024-09-22T03:24:59.127722+0000","last_undegraded":"2024-09-22T03:24:59.127722+0000","last_fullsized":"2024-09-22T03:24:59.127722+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29425664,"num_objects":449,"num_object_clones":0,"num_object_copies":1347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":449,"num_write_kb":28736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.125507+0000","last_change":"2024-09-22T03:24:50.129335+0000","last_active":"2024-09-22T03:24:50.125507+0000","last_peered":"2024-09-22T03:24:50.125507+0000","last_clean":"2024-09-22T03:24:50.125507+0000","last_became_active":"2024-09-22T03:24:49.229324+0000","last_became_peered":"2024-09-22T03:24:49.229324+0000","last_unstale":"2024-09-22T03:24:50.125507+0000","last_undegraded":"2024-09-22T03:24:50.125507+0000","last_fullsized":"2024-09-22T03:24:50.125507+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"174'414","reported_seq":440,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:57.102121+0000","last_change":"2024-09-22T03:24:49.226520+0000","last_active":"2024-09-22T03:24:57.102121+0000","last_peered":"2024-09-22T03:24:57.102121+0000","last_clean":"2024-09-22T03:24:57.102121+0000","last_became_active":"2024-09-22T03:24:49.222276+0000","last_became_peered":"2024-09-22T03:24:49.222276+0000","last_unstale":"2024-09-22T03:24:57.102121+0000","last_undegraded":"2024-09-22T03:24:57.102121+0000","last_fullsized":"2024-09-22T03:24:57.102121+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":26560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":336,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.132373+0000","last_change":"2024-09-22T03:24:50.146187+0000","last_active":"2024-09-22T03:24:50.132373+0000","last_peered":"2024-09-22T03:24:50.132373+0000","last_clean":"2024-09-22T03:24:50.132373+0000","last_became_active":"2024-09-22T03:24:49.269404+0000","last_became_peered":"2024-09-22T03:24:49.269404+0000","last_unstale":"2024-09-22T03:24:50.132373+0000","last_undegraded":"2024-09-22T03:24:50.132373+0000","last_fullsized":"2024-09-22T03:24:50.132373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"174'438","reported_seq":463,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:57.968211+0000","last_change":"2024-09-22T03:24:49.265910+0000","last_active":"2024-09-22T03:24:57.968211+0000","last_peered":"2024-09-22T03:24:57.968211+0000","last_clean":"2024-09-22T03:24:57.968211+0000","last_became_active":"2024-09-22T03:24:49.260555+0000","last_became_peered":"2024-09-22T03:24:49.260555+0000","last_unstale":"2024-09-22T03:24:57.968211+0000","last_undegraded":"2024-09-22T03:24:57.968211+0000","last_fullsized":"2024-09-22T03:24:57.968211+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":439,"num_object_clones":0,"num_object_copies":1317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":439,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":317,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.127634+0000","last_change":"2024-09-22T03:24:50.130812+0000","last_active":"2024-09-22T03:24:50.127634+0000","last_peered":"2024-09-22T03:24:50.127634+0000","last_clean":"2024-09-22T03:24:50.127634+0000","last_became_active":"2024-09-22T03:24:49.246771+0000","last_became_peered":"2024-09-22T03:24:49.246771+0000","last_unstale":"2024-09-22T03:24:50.127634+0000","last_undegraded":"2024-09-22T03:24:50.127634+0000","last_fullsized":"2024-09-22T03:24:50.127634+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"174'433","reported_seq":458,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:58.040734+0000","last_change":"2024-09-22T03:24:49.258029+0000","last_active":"2024-09-22T03:24:58.040734+0000","last_peered":"2024-09-22T03:24:58.040734+0000","last_clean":"2024-09-22T03:24:58.040734+0000","last_became_active":"2024-09-22T03:24:49.251533+0000","last_became_peered":"2024-09-22T03:24:49.251533+0000","last_unstale":"2024-09-22T03:24:58.040734+0000","last_undegraded":"2024-09-22T03:24:58.040734+0000","last_fullsized":"2024-09-22T03:24:58.040734+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":434,"num_write_kb":27776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":270,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.123717+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:50.123717+0000","last_peered":"2024-09-22T03:24:50.123717+0000","last_clean":"2024-09-22T03:24:50.123717+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:50.123717+0000","last_undegraded":"2024-09-22T03:24:50.123717+0000","last_fullsized":"2024-09-22T03:24:50.123717+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":446889984,"num_objects":6819,"num_object_clones":0,"num_object_copies":20457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6819,"num_write_kb":436416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":84180992,"data_stored":1346895872,"data_compressed":63954494,"data_compressed_allocated":84180992,"data_compressed_original":1346895872,"omap_allocated":0,"internal_metadata":0},"log_size":6803,"ondisk_log_size":6803,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69592,"kb_used_data":19336,"kb_used_omap":0,"kb_used_meta":50240,"kb_avail":94302248,"statfs":{"total":96636764160,"available":96565501952,"internally_reserved":0,"allocated":19800064,"data_stored":294260922,"data_compressed":13944116,"data_compressed_allocated":18354176,"data_compressed_original":293666816,"omap_allocated":0,"internal_metadata":51445760},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":172,"seq":738734374921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63664,"kb_used_data":19424,"kb_used_omap":0,"kb_used_meta":44224,"kb_avail":94308176,"statfs":{"total":96636764160,"available":96571572224,"internally_reserved":0,"allocated":19890176,"data_stored":295310042,"data_compressed":13995881,"data_compressed_allocated":18423808,"data_compressed_original":294699008,"omap_allocated":0,"internal_metadata":45285376},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800119,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60672,"kb_used_data":21424,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94311168,"statfs":{"total":96636764160,"available":96574636032,"internally_reserved":0,"allocated":21938176,"data_stored":328078042,"data_compressed":15551710,"data_compressed_allocated":20471808,"data_compressed_original":327467008,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611828032,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106728,"kb_used_data":24152,"kb_used_omap":0,"kb_used_meta":82560,"kb_avail":94265112,"statfs":{"total":96636764160,"available":96527474688,"internally_reserved":0,"allocated":24731648,"data_stored":372839111,"data_compressed":17677215,"data_compressed_allocated":23269376,"data_compressed_original":372228096,"omap_allocated":0,"internal_metadata":84541440},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23932928,"data_stored":382926848,"data_compressed":18182507,"data_compressed_allocated":23932928,"data_compressed_original":382926848,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21630976,"data_stored":346095616,"data_compressed":16433554,"data_compressed_allocated":21630976,"data_compressed_original":346095616,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19402752,"data_stored":310444032,"data_compressed":14740797,"data_compressed_allocated":19402752,"data_compressed_original":310444032,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19214336,"data_stored":307429376,"data_compressed":14597636,"data_compressed_allocated":19214336,"data_compressed_original":307429376,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:03.525 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:03.541 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 42 16 8877 8861 13.1846 22.9375 0.0374125 0.0758009 2024-09-22T03:25:03.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:03.677+0000 7efd76397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:03.688+0000 7efd76397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:04.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 43 16 9256 9240 13.4288 23.6875 0.057473 0.0744228 2024-09-22T03:25:05.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 44 16 9624 9608 13.6463 23 0.0328378 0.0732416 2024-09-22T03:25:05.943 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:06.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:06.089+0000 7f38a8797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:06.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:06.097+0000 7f38a8797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:06.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 45 16 9984 9968 13.843 22.5 0.0403884 0.0721993 2024-09-22T03:25:07.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 46 16 10354 10338 14.0447 23.125 0.0142508 0.0711491 2024-09-22T03:25:07.632 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:25:07.632 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:07.632 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:07.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:07.785+0000 7ff0a046c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:07.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:07.797+0000 7ff0a046c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:08.147 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:25:08.148 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:25:08.165 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2024-09-22T03:25:07.403279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602407475,"num_objects":9187,"num_object_clones":0,"num_object_copies":27561,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9322,"num_write_kb":589608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9267,"ondisk_log_size":9267,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":339004,"kb_used_data":112956,"kb_used_omap":0,"kb_used_meta":225984,"kb_avail":377148356,"statfs":{"total":386547056640,"available":386199916544,"internally_reserved":0,"allocated":115666944,"data_stored":1759398197,"data_compressed":83435282,"data_compressed_allocated":109826048,"data_compressed_original":1756971008,"omap_allocated":0,"internal_metadata":231407616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":280690688,"num_objects":4283,"num_object_clones":0,"num_object_copies":12849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4283,"num_write_kb":274112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002027"},"pg_stats":[{"pgid":"6.f","version":"174'560","reported_seq":585,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:03.040042+0000","last_change":"2024-09-22T03:24:49.284049+0000","last_active":"2024-09-22T03:25:03.040042+0000","last_peered":"2024-09-22T03:25:03.040042+0000","last_clean":"2024-09-22T03:25:03.040042+0000","last_became_active":"2024-09-22T03:24:49.277059+0000","last_became_peered":"2024-09-22T03:24:49.277059+0000","last_unstale":"2024-09-22T03:25:03.040042+0000","last_undegraded":"2024-09-22T03:25:03.040042+0000","last_fullsized":"2024-09-22T03:25:03.040042+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":561,"num_object_clones":0,"num_object_copies":1683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":561,"num_write_kb":35904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"174'650","reported_seq":676,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:07.036374+0000","last_change":"2024-09-22T03:24:49.243863+0000","last_active":"2024-09-22T03:25:07.036374+0000","last_peered":"2024-09-22T03:25:07.036374+0000","last_clean":"2024-09-22T03:25:07.036374+0000","last_became_active":"2024-09-22T03:24:49.238475+0000","last_became_peered":"2024-09-22T03:24:49.238475+0000","last_unstale":"2024-09-22T03:25:07.036374+0000","last_undegraded":"2024-09-22T03:25:07.036374+0000","last_fullsized":"2024-09-22T03:25:07.036374+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42663936,"num_objects":651,"num_object_clones":0,"num_object_copies":1953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":651,"num_write_kb":41664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"174'608","reported_seq":633,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:07.057830+0000","last_change":"2024-09-22T03:24:20.582250+0000","last_active":"2024-09-22T03:25:07.057830+0000","last_peered":"2024-09-22T03:25:07.057830+0000","last_clean":"2024-09-22T03:25:07.057830+0000","last_became_active":"2024-09-22T03:24:20.575484+0000","last_became_peered":"2024-09-22T03:24:20.575484+0000","last_unstale":"2024-09-22T03:25:07.057830+0000","last_undegraded":"2024-09-22T03:25:07.057830+0000","last_fullsized":"2024-09-22T03:25:07.057830+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":609,"num_object_clones":0,"num_object_copies":1827,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":38976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"174'567","reported_seq":602,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:03.124630+0000","last_change":"2024-09-22T03:24:49.377720+0000","last_active":"2024-09-22T03:25:03.124630+0000","last_peered":"2024-09-22T03:25:03.124630+0000","last_clean":"2024-09-22T03:25:03.124630+0000","last_became_active":"2024-09-22T03:24:49.227549+0000","last_became_peered":"2024-09-22T03:24:49.227549+0000","last_unstale":"2024-09-22T03:25:03.124630+0000","last_undegraded":"2024-09-22T03:25:03.124630+0000","last_fullsized":"2024-09-22T03:25:03.124630+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37224448,"num_objects":568,"num_object_clones":0,"num_object_copies":1704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":36352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"174'660","reported_seq":686,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:06.990888+0000","last_change":"2024-09-22T03:24:49.219983+0000","last_active":"2024-09-22T03:25:06.990888+0000","last_peered":"2024-09-22T03:25:06.990888+0000","last_clean":"2024-09-22T03:25:06.990888+0000","last_became_active":"2024-09-22T03:24:49.214900+0000","last_became_peered":"2024-09-22T03:24:49.214900+0000","last_unstale":"2024-09-22T03:25:06.990888+0000","last_undegraded":"2024-09-22T03:25:06.990888+0000","last_fullsized":"2024-09-22T03:25:06.990888+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43319296,"num_objects":661,"num_object_clones":0,"num_object_copies":1983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":661,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"174'529","reported_seq":554,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:02.948239+0000","last_change":"2024-09-22T03:24:49.253782+0000","last_active":"2024-09-22T03:25:02.948239+0000","last_peered":"2024-09-22T03:25:02.948239+0000","last_clean":"2024-09-22T03:25:02.948239+0000","last_became_active":"2024-09-22T03:24:49.245555+0000","last_became_peered":"2024-09-22T03:24:49.245555+0000","last_unstale":"2024-09-22T03:25:02.948239+0000","last_undegraded":"2024-09-22T03:25:02.948239+0000","last_fullsized":"2024-09-22T03:25:02.948239+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34734080,"num_objects":530,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":530,"num_write_kb":33920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"174'553","reported_seq":590,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:03.075190+0000","last_change":"2024-09-22T03:24:49.402389+0000","last_active":"2024-09-22T03:25:03.075190+0000","last_peered":"2024-09-22T03:25:03.075190+0000","last_clean":"2024-09-22T03:25:03.075190+0000","last_became_active":"2024-09-22T03:24:49.215003+0000","last_became_peered":"2024-09-22T03:24:49.215003+0000","last_unstale":"2024-09-22T03:25:03.075190+0000","last_undegraded":"2024-09-22T03:25:03.075190+0000","last_fullsized":"2024-09-22T03:25:03.075190+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36306944,"num_objects":554,"num_object_clones":0,"num_object_copies":1662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":35456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"174'549","reported_seq":580,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.207841+0000","last_change":"2024-09-22T03:24:49.337699+0000","last_active":"2024-09-22T03:25:04.207841+0000","last_peered":"2024-09-22T03:25:04.207841+0000","last_clean":"2024-09-22T03:25:04.207841+0000","last_became_active":"2024-09-22T03:24:49.238244+0000","last_became_peered":"2024-09-22T03:24:49.238244+0000","last_unstale":"2024-09-22T03:25:04.207841+0000","last_undegraded":"2024-09-22T03:25:04.207841+0000","last_fullsized":"2024-09-22T03:25:04.207841+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36044800,"num_objects":550,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":550,"num_write_kb":35200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"174'548","reported_seq":573,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.189875+0000","last_change":"2024-09-22T03:24:20.529927+0000","last_active":"2024-09-22T03:25:04.189875+0000","last_peered":"2024-09-22T03:25:04.189875+0000","last_clean":"2024-09-22T03:25:04.189875+0000","last_became_active":"2024-09-22T03:24:20.524882+0000","last_became_peered":"2024-09-22T03:24:20.524882+0000","last_unstale":"2024-09-22T03:25:04.189875+0000","last_undegraded":"2024-09-22T03:25:04.189875+0000","last_fullsized":"2024-09-22T03:25:04.189875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":549,"num_object_clones":0,"num_object_copies":1647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":549,"num_write_kb":35136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":300,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131545+0000","last_change":"2024-09-22T03:24:50.133990+0000","last_active":"2024-09-22T03:24:50.131545+0000","last_peered":"2024-09-22T03:24:50.131545+0000","last_clean":"2024-09-22T03:24:50.131545+0000","last_became_active":"2024-09-22T03:24:49.230660+0000","last_became_peered":"2024-09-22T03:24:49.230660+0000","last_unstale":"2024-09-22T03:24:50.131545+0000","last_undegraded":"2024-09-22T03:24:50.131545+0000","last_fullsized":"2024-09-22T03:24:50.131545+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"174'534","reported_seq":559,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:03.129182+0000","last_change":"2024-09-22T03:24:20.543343+0000","last_active":"2024-09-22T03:25:03.129182+0000","last_peered":"2024-09-22T03:25:03.129182+0000","last_clean":"2024-09-22T03:25:03.129182+0000","last_became_active":"2024-09-22T03:24:20.537036+0000","last_became_peered":"2024-09-22T03:24:20.537036+0000","last_unstale":"2024-09-22T03:25:03.129182+0000","last_undegraded":"2024-09-22T03:25:03.129182+0000","last_fullsized":"2024-09-22T03:25:03.129182+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":535,"num_object_clones":0,"num_object_copies":1605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":535,"num_write_kb":34240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":420,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131531+0000","last_change":"2024-09-22T03:24:50.133761+0000","last_active":"2024-09-22T03:24:50.131531+0000","last_peered":"2024-09-22T03:24:50.131531+0000","last_clean":"2024-09-22T03:24:50.131531+0000","last_became_active":"2024-09-22T03:24:49.245678+0000","last_became_peered":"2024-09-22T03:24:49.245678+0000","last_unstale":"2024-09-22T03:24:50.131531+0000","last_undegraded":"2024-09-22T03:24:50.131531+0000","last_fullsized":"2024-09-22T03:24:50.131531+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"174'547","reported_seq":578,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:03.001711+0000","last_change":"2024-09-22T03:24:49.407992+0000","last_active":"2024-09-22T03:25:03.001711+0000","last_peered":"2024-09-22T03:25:03.001711+0000","last_clean":"2024-09-22T03:25:03.001711+0000","last_became_active":"2024-09-22T03:24:49.262959+0000","last_became_peered":"2024-09-22T03:24:49.262959+0000","last_unstale":"2024-09-22T03:25:03.001711+0000","last_undegraded":"2024-09-22T03:25:03.001711+0000","last_fullsized":"2024-09-22T03:25:03.001711+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":548,"num_object_clones":0,"num_object_copies":1644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":548,"num_write_kb":35072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":436,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.135967+0000","last_change":"2024-09-22T03:24:50.139730+0000","last_active":"2024-09-22T03:24:50.135967+0000","last_peered":"2024-09-22T03:24:50.135967+0000","last_clean":"2024-09-22T03:24:50.135967+0000","last_became_active":"2024-09-22T03:24:49.263866+0000","last_became_peered":"2024-09-22T03:24:49.263866+0000","last_unstale":"2024-09-22T03:24:50.135967+0000","last_undegraded":"2024-09-22T03:24:50.135967+0000","last_fullsized":"2024-09-22T03:24:50.135967+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"174'537","reported_seq":562,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:02.894026+0000","last_change":"2024-09-22T03:24:20.508982+0000","last_active":"2024-09-22T03:25:02.894026+0000","last_peered":"2024-09-22T03:25:02.894026+0000","last_clean":"2024-09-22T03:25:02.894026+0000","last_became_active":"2024-09-22T03:24:20.505162+0000","last_became_peered":"2024-09-22T03:24:20.505162+0000","last_unstale":"2024-09-22T03:25:02.894026+0000","last_undegraded":"2024-09-22T03:25:02.894026+0000","last_fullsized":"2024-09-22T03:25:02.894026+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258368,"num_objects":538,"num_object_clones":0,"num_object_copies":1614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":538,"num_write_kb":34432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":380,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.130593+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:50.130593+0000","last_peered":"2024-09-22T03:24:50.130593+0000","last_clean":"2024-09-22T03:24:50.130593+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:50.130593+0000","last_undegraded":"2024-09-22T03:24:50.130593+0000","last_fullsized":"2024-09-22T03:24:50.130593+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":538,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131988+0000","last_change":"2024-09-22T03:24:49.229480+0000","last_active":"2024-09-22T03:24:50.131988+0000","last_peered":"2024-09-22T03:24:50.131988+0000","last_clean":"2024-09-22T03:24:50.131988+0000","last_became_active":"2024-09-22T03:24:49.225634+0000","last_became_peered":"2024-09-22T03:24:49.225634+0000","last_unstale":"2024-09-22T03:24:50.131988+0000","last_undegraded":"2024-09-22T03:24:50.131988+0000","last_fullsized":"2024-09-22T03:24:50.131988+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"174'581","reported_seq":606,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.100105+0000","last_change":"2024-09-22T03:24:20.555078+0000","last_active":"2024-09-22T03:25:04.100105+0000","last_peered":"2024-09-22T03:25:04.100105+0000","last_clean":"2024-09-22T03:25:04.100105+0000","last_became_active":"2024-09-22T03:24:20.549566+0000","last_became_peered":"2024-09-22T03:24:20.549566+0000","last_unstale":"2024-09-22T03:25:04.100105+0000","last_undegraded":"2024-09-22T03:25:04.100105+0000","last_fullsized":"2024-09-22T03:25:04.100105+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":582,"num_object_clones":0,"num_object_copies":1746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":582,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.125507+0000","last_change":"2024-09-22T03:24:50.129335+0000","last_active":"2024-09-22T03:24:50.125507+0000","last_peered":"2024-09-22T03:24:50.125507+0000","last_clean":"2024-09-22T03:24:50.125507+0000","last_became_active":"2024-09-22T03:24:49.229324+0000","last_became_peered":"2024-09-22T03:24:49.229324+0000","last_unstale":"2024-09-22T03:24:50.125507+0000","last_undegraded":"2024-09-22T03:24:50.125507+0000","last_fullsized":"2024-09-22T03:24:50.125507+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"174'646","reported_seq":672,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:07.054407+0000","last_change":"2024-09-22T03:24:49.226520+0000","last_active":"2024-09-22T03:25:07.054407+0000","last_peered":"2024-09-22T03:25:07.054407+0000","last_clean":"2024-09-22T03:25:07.054407+0000","last_became_active":"2024-09-22T03:24:49.222276+0000","last_became_peered":"2024-09-22T03:24:49.222276+0000","last_unstale":"2024-09-22T03:25:07.054407+0000","last_undegraded":"2024-09-22T03:25:07.054407+0000","last_fullsized":"2024-09-22T03:25:07.054407+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42401792,"num_objects":647,"num_object_clones":0,"num_object_copies":1941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":647,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":336,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.132373+0000","last_change":"2024-09-22T03:24:50.146187+0000","last_active":"2024-09-22T03:24:50.132373+0000","last_peered":"2024-09-22T03:24:50.132373+0000","last_clean":"2024-09-22T03:24:50.132373+0000","last_became_active":"2024-09-22T03:24:49.269404+0000","last_became_peered":"2024-09-22T03:24:49.269404+0000","last_unstale":"2024-09-22T03:24:50.132373+0000","last_undegraded":"2024-09-22T03:24:50.132373+0000","last_fullsized":"2024-09-22T03:24:50.132373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"174'559","reported_seq":584,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:03.048179+0000","last_change":"2024-09-22T03:24:49.265910+0000","last_active":"2024-09-22T03:25:03.048179+0000","last_peered":"2024-09-22T03:25:03.048179+0000","last_clean":"2024-09-22T03:25:03.048179+0000","last_became_active":"2024-09-22T03:24:49.260555+0000","last_became_peered":"2024-09-22T03:24:49.260555+0000","last_unstale":"2024-09-22T03:25:03.048179+0000","last_undegraded":"2024-09-22T03:25:03.048179+0000","last_fullsized":"2024-09-22T03:25:03.048179+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36700160,"num_objects":560,"num_object_clones":0,"num_object_copies":1680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":35840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":317,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.127634+0000","last_change":"2024-09-22T03:24:50.130812+0000","last_active":"2024-09-22T03:24:50.127634+0000","last_peered":"2024-09-22T03:24:50.127634+0000","last_clean":"2024-09-22T03:24:50.127634+0000","last_became_active":"2024-09-22T03:24:49.246771+0000","last_became_peered":"2024-09-22T03:24:49.246771+0000","last_unstale":"2024-09-22T03:24:50.127634+0000","last_undegraded":"2024-09-22T03:24:50.127634+0000","last_fullsized":"2024-09-22T03:24:50.127634+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"174'539","reported_seq":564,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:03.015438+0000","last_change":"2024-09-22T03:24:49.258029+0000","last_active":"2024-09-22T03:25:03.015438+0000","last_peered":"2024-09-22T03:25:03.015438+0000","last_clean":"2024-09-22T03:25:03.015438+0000","last_became_active":"2024-09-22T03:24:49.251533+0000","last_became_peered":"2024-09-22T03:24:49.251533+0000","last_unstale":"2024-09-22T03:25:03.015438+0000","last_undegraded":"2024-09-22T03:25:03.015438+0000","last_fullsized":"2024-09-22T03:25:03.015438+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":540,"num_write_kb":34560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":270,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.123717+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:50.123717+0000","last_peered":"2024-09-22T03:24:50.123717+0000","last_clean":"2024-09-22T03:24:50.123717+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:50.123717+0000","last_undegraded":"2024-09-22T03:24:50.123717+0000","last_fullsized":"2024-09-22T03:24:50.123717+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":601817088,"num_objects":9183,"num_object_clones":0,"num_object_copies":27549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9183,"num_write_kb":587712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":113442816,"data_stored":1815085056,"data_compressed":86186622,"data_compressed_allocated":113442816,"data_compressed_original":1815085056,"omap_allocated":0,"internal_metadata":0},"log_size":9167,"ondisk_log_size":9167,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599072,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96960,"kb_used_data":24496,"kb_used_omap":0,"kb_used_meta":72448,"kb_avail":94274880,"statfs":{"total":96636764160,"available":96537477120,"internally_reserved":0,"allocated":25083904,"data_stored":378802362,"data_compressed":17958596,"data_compressed_allocated":23638016,"data_compressed_original":378208256,"omap_allocated":0,"internal_metadata":74186752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":172,"seq":738734374926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56376,"kb_used_data":24616,"kb_used_omap":0,"kb_used_meta":31744,"kb_avail":94315464,"statfs":{"total":96636764160,"available":96579035136,"internally_reserved":0,"allocated":25206784,"data_stored":380375770,"data_compressed":18035257,"data_compressed_allocated":23740416,"data_compressed_original":379764736,"omap_allocated":0,"internal_metadata":32505856},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800129,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81164,"kb_used_data":33596,"kb_used_omap":0,"kb_used_meta":47552,"kb_avail":94290676,"statfs":{"total":96636764160,"available":96553652224,"internally_reserved":0,"allocated":34402304,"data_stored":527504090,"data_compressed":25021526,"data_compressed_allocated":32935936,"data_compressed_original":526893056,"omap_allocated":0,"internal_metadata":48693248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611828037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":104504,"kb_used_data":30248,"kb_used_omap":0,"kb_used_meta":74240,"kb_avail":94267336,"statfs":{"total":96636764160,"available":96529752064,"internally_reserved":0,"allocated":30973952,"data_stored":472715975,"data_compressed":22419903,"data_compressed_allocated":29511680,"data_compressed_original":472104960,"omap_allocated":0,"internal_metadata":76021760},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30244864,"data_stored":483917824,"data_compressed":22978099,"data_compressed_allocated":30244864,"data_compressed_original":483917824,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33980416,"data_stored":543686656,"data_compressed":25816234,"data_compressed_allocated":33980416,"data_compressed_original":543686656,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24784896,"data_stored":396558336,"data_compressed":18829965,"data_compressed_allocated":24784896,"data_compressed_original":396558336,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24432640,"data_stored":390922240,"data_compressed":18562324,"data_compressed_allocated":24432640,"data_compressed_original":390922240,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:08.297 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:08.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:08.449+0000 7ff259e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:08.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:08.456+0000 7ff259e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:08.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 47 16 10707 10691 14.2153 22.0625 0.0254657 0.0703117 2024-09-22T03:25:09.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 48 16 11059 11043 14.3774 22 0.0312444 0.0695284 2024-09-22T03:25:10.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 49 16 11415 11399 14.5381 22.25 0.0423891 0.0687541 2024-09-22T03:25:10.628 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:10.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:10.757+0000 7f2a03957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:10.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:10.765+0000 7f2a03957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:11.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 50 16 11771 11755 14.6923 22.25 0.0435318 0.068027 2024-09-22T03:25:12.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 51 16 12137 12121 14.8527 22.875 0.0740951 0.0672918 2024-09-22T03:25:12.983 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:13.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.149+0000 7f03a68d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.158+0000 7f03a5675640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.167 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:25:13.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:13.167 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:25:13.168 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:25:13.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.323+0000 7fc375ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.334+0000 7fc375ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.370 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:13.370 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:25:13.370 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:25:13.370 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:25:13.370 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:25:13.370 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:25:13.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:25:13.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:25:13.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:25:13.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:25:13.371 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:25:13.371 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:25:13.383 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:13.383 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:13.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:25:13.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 52 16 12472 12456 14.9696 20.9375 0.0513724 0.0667533 2024-09-22T03:25:13.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.578+0000 7f9b174ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.593+0000 7f9b174ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.644 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:25:13.645 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:25:13.646 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:25:13.650 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:13.651 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:13.651 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:25:13.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.794+0000 7f3d27a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:13.805+0000 7f3d27a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:25:13.887 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:25:13.888 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:25:13.890 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:13.890 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:13.891 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:25:14.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:14.028+0000 7f62ebd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:14.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:14.038+0000 7f62ebd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:25:14.055 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:25:14.056 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:25:14.059 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:14.059 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:14.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 53 16 12792 12776 15.0645 20 0.0749949 0.066348 2024-09-22T03:25:15.385 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:15.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:15.532+0000 7f27ec197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.542 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 54 16 13141 13125 15.1894 21.8125 0.041958 0.0658087 2024-09-22T03:25:15.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:15.542+0000 7f27ec197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:16.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 55 16 13484 13468 15.303 21.4375 0.0633492 0.0653088 2024-09-22T03:25:17.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 56 16 13838 13822 15.4248 22.125 0.043926 0.0647897 2024-09-22T03:25:17.723 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:17.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:17.884+0000 7f93bd397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:17.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:17.893+0000 7f93bd397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:18.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 57 16 14180 14164 15.5292 21.375 0.0535097 0.06436 2024-09-22T03:25:19.060 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:25:19.060 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:19.060 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:19.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:19.195+0000 7f6630357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:19.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:19.211+0000 7f6630357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:19.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 58 16 14512 14496 15.6191 20.75 0.0359359 0.0639847 2024-09-22T03:25:19.574 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:25:19.575 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:25:19.592 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2024-09-22T03:25:19.405454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":919470643,"num_objects":14025,"num_object_clones":0,"num_object_copies":42075,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":14160,"num_write_kb":899240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14105,"ondisk_log_size":14105,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":392840,"kb_used_data":170948,"kb_used_omap":0,"kb_used_meta":221824,"kb_avail":377094520,"statfs":{"total":386547056640,"available":386144788480,"internally_reserved":0,"allocated":175050752,"data_stored":2709539125,"data_compressed":128553055,"data_compressed_allocated":169209856,"data_compressed_original":2707111936,"omap_allocated":0,"internal_metadata":227147776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":317063168,"num_objects":4838,"num_object_clones":0,"num_object_copies":14514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4838,"num_write_kb":309632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002174"},"pg_stats":[{"pgid":"6.f","version":"174'884","reported_seq":909,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.038926+0000","last_change":"2024-09-22T03:24:49.284049+0000","last_active":"2024-09-22T03:25:18.038926+0000","last_peered":"2024-09-22T03:25:18.038926+0000","last_clean":"2024-09-22T03:25:18.038926+0000","last_became_active":"2024-09-22T03:24:49.277059+0000","last_became_peered":"2024-09-22T03:24:49.277059+0000","last_unstale":"2024-09-22T03:25:18.038926+0000","last_undegraded":"2024-09-22T03:25:18.038926+0000","last_fullsized":"2024-09-22T03:25:18.038926+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57999360,"num_objects":885,"num_object_clones":0,"num_object_copies":2655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":885,"num_write_kb":56640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"174'857","reported_seq":883,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:17.094125+0000","last_change":"2024-09-22T03:24:49.243863+0000","last_active":"2024-09-22T03:25:17.094125+0000","last_peered":"2024-09-22T03:25:17.094125+0000","last_clean":"2024-09-22T03:25:17.094125+0000","last_became_active":"2024-09-22T03:24:49.238475+0000","last_became_peered":"2024-09-22T03:24:49.238475+0000","last_unstale":"2024-09-22T03:25:17.094125+0000","last_undegraded":"2024-09-22T03:25:17.094125+0000","last_fullsized":"2024-09-22T03:25:17.094125+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":858,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"174'824","reported_seq":849,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:17.084801+0000","last_change":"2024-09-22T03:24:20.582250+0000","last_active":"2024-09-22T03:25:17.084801+0000","last_peered":"2024-09-22T03:25:17.084801+0000","last_clean":"2024-09-22T03:25:17.084801+0000","last_became_active":"2024-09-22T03:24:20.575484+0000","last_became_peered":"2024-09-22T03:24:20.575484+0000","last_unstale":"2024-09-22T03:25:17.084801+0000","last_undegraded":"2024-09-22T03:25:17.084801+0000","last_fullsized":"2024-09-22T03:25:17.084801+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54067200,"num_objects":825,"num_object_clones":0,"num_object_copies":2475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":825,"num_write_kb":52800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"174'913","reported_seq":948,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.116760+0000","last_change":"2024-09-22T03:24:49.377720+0000","last_active":"2024-09-22T03:25:18.116760+0000","last_peered":"2024-09-22T03:25:18.116760+0000","last_clean":"2024-09-22T03:25:18.116760+0000","last_became_active":"2024-09-22T03:24:49.227549+0000","last_became_peered":"2024-09-22T03:24:49.227549+0000","last_unstale":"2024-09-22T03:25:18.116760+0000","last_undegraded":"2024-09-22T03:25:18.116760+0000","last_fullsized":"2024-09-22T03:25:18.116760+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59899904,"num_objects":914,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":914,"num_write_kb":58496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"174'862","reported_seq":888,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:17.038240+0000","last_change":"2024-09-22T03:24:49.219983+0000","last_active":"2024-09-22T03:25:17.038240+0000","last_peered":"2024-09-22T03:25:17.038240+0000","last_clean":"2024-09-22T03:25:17.038240+0000","last_became_active":"2024-09-22T03:24:49.214900+0000","last_became_peered":"2024-09-22T03:24:49.214900+0000","last_unstale":"2024-09-22T03:25:17.038240+0000","last_undegraded":"2024-09-22T03:25:17.038240+0000","last_fullsized":"2024-09-22T03:25:17.038240+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":863,"num_write_kb":55232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"174'876","reported_seq":901,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.050866+0000","last_change":"2024-09-22T03:24:49.253782+0000","last_active":"2024-09-22T03:25:18.050866+0000","last_peered":"2024-09-22T03:25:18.050866+0000","last_clean":"2024-09-22T03:25:18.050866+0000","last_became_active":"2024-09-22T03:24:49.245555+0000","last_became_peered":"2024-09-22T03:24:49.245555+0000","last_unstale":"2024-09-22T03:25:18.050866+0000","last_undegraded":"2024-09-22T03:25:18.050866+0000","last_fullsized":"2024-09-22T03:25:18.050866+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57475072,"num_objects":877,"num_object_clones":0,"num_object_copies":2631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":877,"num_write_kb":56128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"174'852","reported_seq":889,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.133887+0000","last_change":"2024-09-22T03:24:49.402389+0000","last_active":"2024-09-22T03:25:18.133887+0000","last_peered":"2024-09-22T03:25:18.133887+0000","last_clean":"2024-09-22T03:25:18.133887+0000","last_became_active":"2024-09-22T03:24:49.215003+0000","last_became_peered":"2024-09-22T03:24:49.215003+0000","last_unstale":"2024-09-22T03:25:18.133887+0000","last_undegraded":"2024-09-22T03:25:18.133887+0000","last_fullsized":"2024-09-22T03:25:18.133887+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55902208,"num_objects":853,"num_object_clones":0,"num_object_copies":2559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":853,"num_write_kb":54592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"174'850","reported_seq":881,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:19.213486+0000","last_change":"2024-09-22T03:24:49.337699+0000","last_active":"2024-09-22T03:25:19.213486+0000","last_peered":"2024-09-22T03:25:19.213486+0000","last_clean":"2024-09-22T03:25:19.213486+0000","last_became_active":"2024-09-22T03:24:49.238244+0000","last_became_peered":"2024-09-22T03:24:49.238244+0000","last_unstale":"2024-09-22T03:25:19.213486+0000","last_undegraded":"2024-09-22T03:25:19.213486+0000","last_fullsized":"2024-09-22T03:25:19.213486+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55771136,"num_objects":851,"num_object_clones":0,"num_object_copies":2553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":851,"num_write_kb":54464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"174'863","reported_seq":888,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:19.113606+0000","last_change":"2024-09-22T03:24:20.529927+0000","last_active":"2024-09-22T03:25:19.113606+0000","last_peered":"2024-09-22T03:25:19.113606+0000","last_clean":"2024-09-22T03:25:19.113606+0000","last_became_active":"2024-09-22T03:24:20.524882+0000","last_became_peered":"2024-09-22T03:24:20.524882+0000","last_unstale":"2024-09-22T03:25:19.113606+0000","last_undegraded":"2024-09-22T03:25:19.113606+0000","last_fullsized":"2024-09-22T03:25:19.113606+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56623104,"num_objects":864,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":55296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":300,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131545+0000","last_change":"2024-09-22T03:24:50.133990+0000","last_active":"2024-09-22T03:24:50.131545+0000","last_peered":"2024-09-22T03:24:50.131545+0000","last_clean":"2024-09-22T03:24:50.131545+0000","last_became_active":"2024-09-22T03:24:49.230660+0000","last_became_peered":"2024-09-22T03:24:49.230660+0000","last_unstale":"2024-09-22T03:24:50.131545+0000","last_undegraded":"2024-09-22T03:24:50.131545+0000","last_fullsized":"2024-09-22T03:24:50.131545+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"174'900","reported_seq":925,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.034476+0000","last_change":"2024-09-22T03:24:20.543343+0000","last_active":"2024-09-22T03:25:18.034476+0000","last_peered":"2024-09-22T03:25:18.034476+0000","last_clean":"2024-09-22T03:25:18.034476+0000","last_became_active":"2024-09-22T03:24:20.537036+0000","last_became_peered":"2024-09-22T03:24:20.537036+0000","last_unstale":"2024-09-22T03:25:18.034476+0000","last_undegraded":"2024-09-22T03:25:18.034476+0000","last_fullsized":"2024-09-22T03:25:18.034476+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59047936,"num_objects":901,"num_object_clones":0,"num_object_copies":2703,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":901,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":420,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131531+0000","last_change":"2024-09-22T03:24:50.133761+0000","last_active":"2024-09-22T03:24:50.131531+0000","last_peered":"2024-09-22T03:24:50.131531+0000","last_clean":"2024-09-22T03:24:50.131531+0000","last_became_active":"2024-09-22T03:24:49.245678+0000","last_became_peered":"2024-09-22T03:24:49.245678+0000","last_unstale":"2024-09-22T03:24:50.131531+0000","last_undegraded":"2024-09-22T03:24:50.131531+0000","last_fullsized":"2024-09-22T03:24:50.131531+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"174'891","reported_seq":922,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.060878+0000","last_change":"2024-09-22T03:24:49.407992+0000","last_active":"2024-09-22T03:25:18.060878+0000","last_peered":"2024-09-22T03:25:18.060878+0000","last_clean":"2024-09-22T03:25:18.060878+0000","last_became_active":"2024-09-22T03:24:49.262959+0000","last_became_peered":"2024-09-22T03:24:49.262959+0000","last_unstale":"2024-09-22T03:25:18.060878+0000","last_undegraded":"2024-09-22T03:25:18.060878+0000","last_fullsized":"2024-09-22T03:25:18.060878+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58458112,"num_objects":892,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":892,"num_write_kb":57088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":436,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.135967+0000","last_change":"2024-09-22T03:24:50.139730+0000","last_active":"2024-09-22T03:24:50.135967+0000","last_peered":"2024-09-22T03:24:50.135967+0000","last_clean":"2024-09-22T03:24:50.135967+0000","last_became_active":"2024-09-22T03:24:49.263866+0000","last_became_peered":"2024-09-22T03:24:49.263866+0000","last_unstale":"2024-09-22T03:24:50.135967+0000","last_undegraded":"2024-09-22T03:24:50.135967+0000","last_fullsized":"2024-09-22T03:24:50.135967+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"174'893","reported_seq":918,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.035454+0000","last_change":"2024-09-22T03:24:20.508982+0000","last_active":"2024-09-22T03:25:18.035454+0000","last_peered":"2024-09-22T03:25:18.035454+0000","last_clean":"2024-09-22T03:25:18.035454+0000","last_became_active":"2024-09-22T03:24:20.505162+0000","last_became_peered":"2024-09-22T03:24:20.505162+0000","last_unstale":"2024-09-22T03:25:18.035454+0000","last_undegraded":"2024-09-22T03:25:18.035454+0000","last_fullsized":"2024-09-22T03:25:18.035454+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58589184,"num_objects":894,"num_object_clones":0,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":894,"num_write_kb":57216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":380,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.130593+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:50.130593+0000","last_peered":"2024-09-22T03:24:50.130593+0000","last_clean":"2024-09-22T03:24:50.130593+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:50.130593+0000","last_undegraded":"2024-09-22T03:24:50.130593+0000","last_fullsized":"2024-09-22T03:24:50.130593+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":538,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131988+0000","last_change":"2024-09-22T03:24:49.229480+0000","last_active":"2024-09-22T03:24:50.131988+0000","last_peered":"2024-09-22T03:24:50.131988+0000","last_clean":"2024-09-22T03:24:50.131988+0000","last_became_active":"2024-09-22T03:24:49.225634+0000","last_became_peered":"2024-09-22T03:24:49.225634+0000","last_unstale":"2024-09-22T03:24:50.131988+0000","last_undegraded":"2024-09-22T03:24:50.131988+0000","last_fullsized":"2024-09-22T03:24:50.131988+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"174'905","reported_seq":930,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:19.211483+0000","last_change":"2024-09-22T03:24:20.555078+0000","last_active":"2024-09-22T03:25:19.211483+0000","last_peered":"2024-09-22T03:25:19.211483+0000","last_clean":"2024-09-22T03:25:19.211483+0000","last_became_active":"2024-09-22T03:24:20.549566+0000","last_became_peered":"2024-09-22T03:24:20.549566+0000","last_unstale":"2024-09-22T03:25:19.211483+0000","last_undegraded":"2024-09-22T03:25:19.211483+0000","last_fullsized":"2024-09-22T03:25:19.211483+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59375616,"num_objects":906,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":906,"num_write_kb":57984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.125507+0000","last_change":"2024-09-22T03:24:50.129335+0000","last_active":"2024-09-22T03:24:50.125507+0000","last_peered":"2024-09-22T03:24:50.125507+0000","last_clean":"2024-09-22T03:24:50.125507+0000","last_became_active":"2024-09-22T03:24:49.229324+0000","last_became_peered":"2024-09-22T03:24:49.229324+0000","last_unstale":"2024-09-22T03:24:50.125507+0000","last_undegraded":"2024-09-22T03:24:50.125507+0000","last_fullsized":"2024-09-22T03:24:50.125507+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"174'847","reported_seq":873,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:16.858975+0000","last_change":"2024-09-22T03:24:49.226520+0000","last_active":"2024-09-22T03:25:16.858975+0000","last_peered":"2024-09-22T03:25:16.858975+0000","last_clean":"2024-09-22T03:25:16.858975+0000","last_became_active":"2024-09-22T03:24:49.222276+0000","last_became_peered":"2024-09-22T03:24:49.222276+0000","last_unstale":"2024-09-22T03:25:16.858975+0000","last_undegraded":"2024-09-22T03:25:16.858975+0000","last_fullsized":"2024-09-22T03:25:16.858975+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55574528,"num_objects":848,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":54272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":336,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.132373+0000","last_change":"2024-09-22T03:24:50.146187+0000","last_active":"2024-09-22T03:24:50.132373+0000","last_peered":"2024-09-22T03:24:50.132373+0000","last_clean":"2024-09-22T03:24:50.132373+0000","last_became_active":"2024-09-22T03:24:49.269404+0000","last_became_peered":"2024-09-22T03:24:49.269404+0000","last_unstale":"2024-09-22T03:24:50.132373+0000","last_undegraded":"2024-09-22T03:24:50.132373+0000","last_fullsized":"2024-09-22T03:24:50.132373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"174'905","reported_seq":930,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:18.090373+0000","last_change":"2024-09-22T03:24:49.265910+0000","last_active":"2024-09-22T03:25:18.090373+0000","last_peered":"2024-09-22T03:25:18.090373+0000","last_clean":"2024-09-22T03:25:18.090373+0000","last_became_active":"2024-09-22T03:24:49.260555+0000","last_became_peered":"2024-09-22T03:24:49.260555+0000","last_unstale":"2024-09-22T03:25:18.090373+0000","last_undegraded":"2024-09-22T03:25:18.090373+0000","last_fullsized":"2024-09-22T03:25:18.090373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59375616,"num_objects":906,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":906,"num_write_kb":57984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":317,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.127634+0000","last_change":"2024-09-22T03:24:50.130812+0000","last_active":"2024-09-22T03:24:50.127634+0000","last_peered":"2024-09-22T03:24:50.127634+0000","last_clean":"2024-09-22T03:24:50.127634+0000","last_became_active":"2024-09-22T03:24:49.246771+0000","last_became_peered":"2024-09-22T03:24:49.246771+0000","last_unstale":"2024-09-22T03:24:50.127634+0000","last_undegraded":"2024-09-22T03:24:50.127634+0000","last_fullsized":"2024-09-22T03:24:50.127634+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"174'883","reported_seq":908,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:17.820167+0000","last_change":"2024-09-22T03:24:49.258029+0000","last_active":"2024-09-22T03:25:17.820167+0000","last_peered":"2024-09-22T03:25:17.820167+0000","last_clean":"2024-09-22T03:25:17.820167+0000","last_became_active":"2024-09-22T03:24:49.251533+0000","last_became_peered":"2024-09-22T03:24:49.251533+0000","last_unstale":"2024-09-22T03:25:17.820167+0000","last_undegraded":"2024-09-22T03:25:17.820167+0000","last_fullsized":"2024-09-22T03:25:17.820167+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57933824,"num_objects":884,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":884,"num_write_kb":56576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":270,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.123717+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:50.123717+0000","last_peered":"2024-09-22T03:24:50.123717+0000","last_clean":"2024-09-22T03:24:50.123717+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:50.123717+0000","last_undegraded":"2024-09-22T03:24:50.123717+0000","last_fullsized":"2024-09-22T03:24:50.123717+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":918880256,"num_objects":14021,"num_object_clones":0,"num_object_copies":42063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14021,"num_write_kb":897344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172425216,"data_stored":2758803456,"data_compressed":130999419,"data_compressed_allocated":172425216,"data_compressed_original":2758803456,"omap_allocated":0,"internal_metadata":0},"log_size":14005,"ondisk_log_size":14005,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599087,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":111608,"kb_used_data":39144,"kb_used_omap":0,"kb_used_meta":72448,"kb_avail":94260232,"statfs":{"total":96636764160,"available":96522477568,"internally_reserved":0,"allocated":40083456,"data_stored":618795194,"data_compressed":29354740,"data_compressed_allocated":38637568,"data_compressed_original":618201088,"omap_allocated":0,"internal_metadata":74186752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":172,"seq":738734374941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71056,"kb_used_data":39292,"kb_used_omap":0,"kb_used_meta":31744,"kb_avail":94300784,"statfs":{"total":96636764160,"available":96564002816,"internally_reserved":0,"allocated":40235008,"data_stored":620827354,"data_compressed":29453184,"data_compressed_allocated":38768640,"data_compressed_original":620216320,"omap_allocated":0,"internal_metadata":32505856},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800139,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88304,"kb_used_data":44896,"kb_used_omap":0,"kb_used_meta":43392,"kb_avail":94283536,"statfs":{"total":96636764160,"available":96546340864,"internally_reserved":0,"allocated":45973504,"data_stored":712643290,"data_compressed":33812925,"data_compressed_allocated":44507136,"data_compressed_original":712032256,"omap_allocated":0,"internal_metadata":44433408},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611828052,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121872,"kb_used_data":47616,"kb_used_omap":0,"kb_used_meta":74240,"kb_avail":94249968,"statfs":{"total":96636764160,"available":96511967232,"internally_reserved":0,"allocated":48758784,"data_stored":757273287,"data_compressed":35932206,"data_compressed_allocated":47296512,"data_compressed_original":756662272,"omap_allocated":0,"internal_metadata":76021760},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":47878144,"data_stored":766050304,"data_compressed":36375258,"data_compressed_allocated":47878144,"data_compressed_original":766050304,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45576192,"data_stored":729219072,"data_compressed":34626305,"data_compressed_allocated":45576192,"data_compressed_original":729219072,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":39612416,"data_stored":633798656,"data_compressed":30095404,"data_compressed_allocated":39612416,"data_compressed_original":633798656,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":39358464,"data_stored":629735424,"data_compressed":29902452,"data_compressed_allocated":39358464,"data_compressed_original":629735424,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:20.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:20.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:20.303+0000 7fbefa6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:20.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:20.309+0000 7fbefa6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:20.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 59 16 14846 14830 15.7082 20.875 0.0599403 0.0636344 2024-09-22T03:25:21.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:25:21.542962+0000 min lat: 0.00860919 max lat: 19.7689 avg lat: 0.0632493 2024-09-22T03:25:21.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:25:21.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 60 16 15184 15168 15.7984 21.125 0.0775699 0.0632493 2024-09-22T03:25:22.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 61 16 15524 15508 15.8878 21.25 0.0487077 0.0629137 2024-09-22T03:25:22.546 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:22.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:22.687+0000 7f2246f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:22.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:22.695+0000 7f2246f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:23.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 62 16 15854 15838 15.9642 20.625 0.0925511 0.062594 2024-09-22T03:25:24.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 63 16 16194 16178 16.048 21.25 0.0491325 0.0622816 2024-09-22T03:25:24.594 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:25:24.594 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:24.594 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:24.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:24.745+0000 7f3cdf957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:24.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:24.761+0000 7f3cdf957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:24.906 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:25.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:25.038+0000 7f40e2957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:25.049 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:25:25.051 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:25:25.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:25.053+0000 7f40e2957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:25.071 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2024-09-22T03:25:23.406201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1009713715,"num_objects":15402,"num_object_clones":0,"num_object_copies":46206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":15537,"num_write_kb":987368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15482,"ondisk_log_size":15482,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":438340,"kb_used_data":185924,"kb_used_omap":0,"kb_used_meta":252352,"kb_avail":377049020,"statfs":{"total":386547056640,"available":386098196480,"internally_reserved":0,"allocated":190386176,"data_stored":2954905909,"data_compressed":140204383,"data_compressed_allocated":184545280,"data_compressed_original":2952478720,"omap_allocated":0,"internal_metadata":258408448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318373888,"num_objects":4858,"num_object_clones":0,"num_object_copies":14574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4858,"num_write_kb":310912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002193"},"pg_stats":[{"pgid":"6.f","version":"174'983","reported_seq":1008,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:22.986892+0000","last_change":"2024-09-22T03:24:49.284049+0000","last_active":"2024-09-22T03:25:22.986892+0000","last_peered":"2024-09-22T03:25:22.986892+0000","last_clean":"2024-09-22T03:25:22.986892+0000","last_became_active":"2024-09-22T03:24:49.277059+0000","last_became_peered":"2024-09-22T03:24:49.277059+0000","last_unstale":"2024-09-22T03:25:22.986892+0000","last_undegraded":"2024-09-22T03:25:22.986892+0000","last_fullsized":"2024-09-22T03:25:22.986892+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64487424,"num_objects":984,"num_object_clones":0,"num_object_copies":2952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":62976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"174'966","reported_seq":992,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:21.974715+0000","last_change":"2024-09-22T03:24:49.243863+0000","last_active":"2024-09-22T03:25:21.974715+0000","last_peered":"2024-09-22T03:25:21.974715+0000","last_clean":"2024-09-22T03:25:21.974715+0000","last_became_active":"2024-09-22T03:24:49.238475+0000","last_became_peered":"2024-09-22T03:24:49.238475+0000","last_unstale":"2024-09-22T03:25:21.974715+0000","last_undegraded":"2024-09-22T03:25:21.974715+0000","last_fullsized":"2024-09-22T03:25:21.974715+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63373312,"num_objects":967,"num_object_clones":0,"num_object_copies":2901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":967,"num_write_kb":61888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"174'925","reported_seq":950,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:21.994104+0000","last_change":"2024-09-22T03:24:20.582250+0000","last_active":"2024-09-22T03:25:21.994104+0000","last_peered":"2024-09-22T03:25:21.994104+0000","last_clean":"2024-09-22T03:25:21.994104+0000","last_became_active":"2024-09-22T03:24:20.575484+0000","last_became_peered":"2024-09-22T03:24:20.575484+0000","last_unstale":"2024-09-22T03:25:21.994104+0000","last_undegraded":"2024-09-22T03:25:21.994104+0000","last_fullsized":"2024-09-22T03:25:21.994104+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60686336,"num_objects":926,"num_object_clones":0,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":926,"num_write_kb":59264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"174'1026","reported_seq":1061,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:23.075437+0000","last_change":"2024-09-22T03:24:49.377720+0000","last_active":"2024-09-22T03:25:23.075437+0000","last_peered":"2024-09-22T03:25:23.075437+0000","last_clean":"2024-09-22T03:25:23.075437+0000","last_became_active":"2024-09-22T03:24:49.227549+0000","last_became_peered":"2024-09-22T03:24:49.227549+0000","last_unstale":"2024-09-22T03:25:23.075437+0000","last_undegraded":"2024-09-22T03:25:23.075437+0000","last_fullsized":"2024-09-22T03:25:23.075437+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67305472,"num_objects":1027,"num_object_clones":0,"num_object_copies":3081,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1027,"num_write_kb":65728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"174'968","reported_seq":994,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:22.081856+0000","last_change":"2024-09-22T03:24:49.219983+0000","last_active":"2024-09-22T03:25:22.081856+0000","last_peered":"2024-09-22T03:25:22.081856+0000","last_clean":"2024-09-22T03:25:22.081856+0000","last_became_active":"2024-09-22T03:24:49.214900+0000","last_became_peered":"2024-09-22T03:24:49.214900+0000","last_unstale":"2024-09-22T03:25:22.081856+0000","last_undegraded":"2024-09-22T03:25:22.081856+0000","last_fullsized":"2024-09-22T03:25:22.081856+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63504384,"num_objects":969,"num_object_clones":0,"num_object_copies":2907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":969,"num_write_kb":62016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"174'972","reported_seq":997,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:23.085819+0000","last_change":"2024-09-22T03:24:49.253782+0000","last_active":"2024-09-22T03:25:23.085819+0000","last_peered":"2024-09-22T03:25:23.085819+0000","last_clean":"2024-09-22T03:25:23.085819+0000","last_became_active":"2024-09-22T03:24:49.245555+0000","last_became_peered":"2024-09-22T03:24:49.245555+0000","last_unstale":"2024-09-22T03:25:23.085819+0000","last_undegraded":"2024-09-22T03:25:23.085819+0000","last_fullsized":"2024-09-22T03:25:23.085819+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63766528,"num_objects":973,"num_object_clones":0,"num_object_copies":2919,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":973,"num_write_kb":62272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"174'960","reported_seq":997,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:22.955892+0000","last_change":"2024-09-22T03:24:49.402389+0000","last_active":"2024-09-22T03:25:22.955892+0000","last_peered":"2024-09-22T03:25:22.955892+0000","last_clean":"2024-09-22T03:25:22.955892+0000","last_became_active":"2024-09-22T03:24:49.215003+0000","last_became_peered":"2024-09-22T03:24:49.215003+0000","last_unstale":"2024-09-22T03:25:22.955892+0000","last_undegraded":"2024-09-22T03:25:22.955892+0000","last_fullsized":"2024-09-22T03:25:22.955892+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62980096,"num_objects":961,"num_object_clones":0,"num_object_copies":2883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":961,"num_write_kb":61504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"174'850","reported_seq":881,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:19.213486+0000","last_change":"2024-09-22T03:24:49.337699+0000","last_active":"2024-09-22T03:25:19.213486+0000","last_peered":"2024-09-22T03:25:19.213486+0000","last_clean":"2024-09-22T03:25:19.213486+0000","last_became_active":"2024-09-22T03:24:49.238244+0000","last_became_peered":"2024-09-22T03:24:49.238244+0000","last_unstale":"2024-09-22T03:25:19.213486+0000","last_undegraded":"2024-09-22T03:25:19.213486+0000","last_fullsized":"2024-09-22T03:25:19.213486+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55771136,"num_objects":851,"num_object_clones":0,"num_object_copies":2553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":851,"num_write_kb":54464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"174'863","reported_seq":888,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:19.113606+0000","last_change":"2024-09-22T03:24:20.529927+0000","last_active":"2024-09-22T03:25:19.113606+0000","last_peered":"2024-09-22T03:25:19.113606+0000","last_clean":"2024-09-22T03:25:19.113606+0000","last_became_active":"2024-09-22T03:24:20.524882+0000","last_became_peered":"2024-09-22T03:24:20.524882+0000","last_unstale":"2024-09-22T03:25:19.113606+0000","last_undegraded":"2024-09-22T03:25:19.113606+0000","last_fullsized":"2024-09-22T03:25:19.113606+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56623104,"num_objects":864,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":55296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":300,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131545+0000","last_change":"2024-09-22T03:24:50.133990+0000","last_active":"2024-09-22T03:24:50.131545+0000","last_peered":"2024-09-22T03:24:50.131545+0000","last_clean":"2024-09-22T03:24:50.131545+0000","last_became_active":"2024-09-22T03:24:49.230660+0000","last_became_peered":"2024-09-22T03:24:49.230660+0000","last_unstale":"2024-09-22T03:24:50.131545+0000","last_undegraded":"2024-09-22T03:24:50.131545+0000","last_fullsized":"2024-09-22T03:24:50.131545+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"174'1027","reported_seq":1052,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:23.101990+0000","last_change":"2024-09-22T03:24:20.543343+0000","last_active":"2024-09-22T03:25:23.101990+0000","last_peered":"2024-09-22T03:25:23.101990+0000","last_clean":"2024-09-22T03:25:23.101990+0000","last_became_active":"2024-09-22T03:24:20.537036+0000","last_became_peered":"2024-09-22T03:24:20.537036+0000","last_unstale":"2024-09-22T03:25:23.101990+0000","last_undegraded":"2024-09-22T03:25:23.101990+0000","last_fullsized":"2024-09-22T03:25:23.101990+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67371008,"num_objects":1028,"num_object_clones":0,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1028,"num_write_kb":65792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":420,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131531+0000","last_change":"2024-09-22T03:24:50.133761+0000","last_active":"2024-09-22T03:24:50.131531+0000","last_peered":"2024-09-22T03:24:50.131531+0000","last_clean":"2024-09-22T03:24:50.131531+0000","last_became_active":"2024-09-22T03:24:49.245678+0000","last_became_peered":"2024-09-22T03:24:49.245678+0000","last_unstale":"2024-09-22T03:24:50.131531+0000","last_undegraded":"2024-09-22T03:24:50.131531+0000","last_fullsized":"2024-09-22T03:24:50.131531+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"174'989","reported_seq":1020,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:23.059124+0000","last_change":"2024-09-22T03:24:49.407992+0000","last_active":"2024-09-22T03:25:23.059124+0000","last_peered":"2024-09-22T03:25:23.059124+0000","last_clean":"2024-09-22T03:25:23.059124+0000","last_became_active":"2024-09-22T03:24:49.262959+0000","last_became_peered":"2024-09-22T03:24:49.262959+0000","last_unstale":"2024-09-22T03:25:23.059124+0000","last_undegraded":"2024-09-22T03:25:23.059124+0000","last_fullsized":"2024-09-22T03:25:23.059124+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64880640,"num_objects":990,"num_object_clones":0,"num_object_copies":2970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":990,"num_write_kb":63360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":436,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.135967+0000","last_change":"2024-09-22T03:24:50.139730+0000","last_active":"2024-09-22T03:24:50.135967+0000","last_peered":"2024-09-22T03:24:50.135967+0000","last_clean":"2024-09-22T03:24:50.135967+0000","last_became_active":"2024-09-22T03:24:49.263866+0000","last_became_peered":"2024-09-22T03:24:49.263866+0000","last_unstale":"2024-09-22T03:24:50.135967+0000","last_undegraded":"2024-09-22T03:24:50.135967+0000","last_fullsized":"2024-09-22T03:24:50.135967+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"174'1007","reported_seq":1032,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:23.090738+0000","last_change":"2024-09-22T03:24:20.508982+0000","last_active":"2024-09-22T03:25:23.090738+0000","last_peered":"2024-09-22T03:25:23.090738+0000","last_clean":"2024-09-22T03:25:23.090738+0000","last_became_active":"2024-09-22T03:24:20.505162+0000","last_became_peered":"2024-09-22T03:24:20.505162+0000","last_unstale":"2024-09-22T03:25:23.090738+0000","last_undegraded":"2024-09-22T03:25:23.090738+0000","last_fullsized":"2024-09-22T03:25:23.090738+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66060288,"num_objects":1008,"num_object_clones":0,"num_object_copies":3024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":64512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":380,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.130593+0000","last_change":"2024-09-22T03:24:03.143711+0000","last_active":"2024-09-22T03:24:50.130593+0000","last_peered":"2024-09-22T03:24:50.130593+0000","last_clean":"2024-09-22T03:24:50.130593+0000","last_became_active":"2024-09-22T03:24:02.289903+0000","last_became_peered":"2024-09-22T03:24:02.289903+0000","last_unstale":"2024-09-22T03:24:50.130593+0000","last_undegraded":"2024-09-22T03:24:50.130593+0000","last_fullsized":"2024-09-22T03:24:50.130593+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":538,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.131988+0000","last_change":"2024-09-22T03:24:49.229480+0000","last_active":"2024-09-22T03:24:50.131988+0000","last_peered":"2024-09-22T03:24:50.131988+0000","last_clean":"2024-09-22T03:24:50.131988+0000","last_became_active":"2024-09-22T03:24:49.225634+0000","last_became_peered":"2024-09-22T03:24:49.225634+0000","last_unstale":"2024-09-22T03:24:50.131988+0000","last_undegraded":"2024-09-22T03:24:50.131988+0000","last_fullsized":"2024-09-22T03:24:50.131988+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"174'905","reported_seq":930,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:19.211483+0000","last_change":"2024-09-22T03:24:20.555078+0000","last_active":"2024-09-22T03:25:19.211483+0000","last_peered":"2024-09-22T03:25:19.211483+0000","last_clean":"2024-09-22T03:25:19.211483+0000","last_became_active":"2024-09-22T03:24:20.549566+0000","last_became_peered":"2024-09-22T03:24:20.549566+0000","last_unstale":"2024-09-22T03:25:19.211483+0000","last_undegraded":"2024-09-22T03:25:19.211483+0000","last_fullsized":"2024-09-22T03:25:19.211483+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59375616,"num_objects":906,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":906,"num_write_kb":57984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.125507+0000","last_change":"2024-09-22T03:24:50.129335+0000","last_active":"2024-09-22T03:24:50.125507+0000","last_peered":"2024-09-22T03:24:50.125507+0000","last_clean":"2024-09-22T03:24:50.125507+0000","last_became_active":"2024-09-22T03:24:49.229324+0000","last_became_peered":"2024-09-22T03:24:49.229324+0000","last_unstale":"2024-09-22T03:24:50.125507+0000","last_undegraded":"2024-09-22T03:24:50.125507+0000","last_fullsized":"2024-09-22T03:24:50.125507+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"174'946","reported_seq":972,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:21.989104+0000","last_change":"2024-09-22T03:24:49.226520+0000","last_active":"2024-09-22T03:25:21.989104+0000","last_peered":"2024-09-22T03:25:21.989104+0000","last_clean":"2024-09-22T03:25:21.989104+0000","last_became_active":"2024-09-22T03:24:49.222276+0000","last_became_peered":"2024-09-22T03:24:49.222276+0000","last_unstale":"2024-09-22T03:25:21.989104+0000","last_undegraded":"2024-09-22T03:25:21.989104+0000","last_fullsized":"2024-09-22T03:25:21.989104+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62062592,"num_objects":947,"num_object_clones":0,"num_object_copies":2841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":947,"num_write_kb":60608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":336,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.132373+0000","last_change":"2024-09-22T03:24:50.146187+0000","last_active":"2024-09-22T03:24:50.132373+0000","last_peered":"2024-09-22T03:24:50.132373+0000","last_clean":"2024-09-22T03:24:50.132373+0000","last_became_active":"2024-09-22T03:24:49.269404+0000","last_became_peered":"2024-09-22T03:24:49.269404+0000","last_unstale":"2024-09-22T03:24:50.132373+0000","last_undegraded":"2024-09-22T03:24:50.132373+0000","last_fullsized":"2024-09-22T03:24:50.132373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"174'1012","reported_seq":1037,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:22.953930+0000","last_change":"2024-09-22T03:24:49.265910+0000","last_active":"2024-09-22T03:25:22.953930+0000","last_peered":"2024-09-22T03:25:22.953930+0000","last_clean":"2024-09-22T03:25:22.953930+0000","last_became_active":"2024-09-22T03:24:49.260555+0000","last_became_peered":"2024-09-22T03:24:49.260555+0000","last_unstale":"2024-09-22T03:25:22.953930+0000","last_undegraded":"2024-09-22T03:25:22.953930+0000","last_fullsized":"2024-09-22T03:25:22.953930+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1012,"log_dups_size":0,"ondisk_log_size":1012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66387968,"num_objects":1013,"num_object_clones":0,"num_object_copies":3039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1013,"num_write_kb":64832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":317,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.127634+0000","last_change":"2024-09-22T03:24:50.130812+0000","last_active":"2024-09-22T03:24:50.127634+0000","last_peered":"2024-09-22T03:24:50.127634+0000","last_clean":"2024-09-22T03:24:50.127634+0000","last_became_active":"2024-09-22T03:24:49.246771+0000","last_became_peered":"2024-09-22T03:24:49.246771+0000","last_unstale":"2024-09-22T03:24:50.127634+0000","last_undegraded":"2024-09-22T03:24:50.127634+0000","last_fullsized":"2024-09-22T03:24:50.127634+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"174'983","reported_seq":1008,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:25:23.031701+0000","last_change":"2024-09-22T03:24:49.258029+0000","last_active":"2024-09-22T03:25:23.031701+0000","last_peered":"2024-09-22T03:25:23.031701+0000","last_clean":"2024-09-22T03:25:23.031701+0000","last_became_active":"2024-09-22T03:24:49.251533+0000","last_became_peered":"2024-09-22T03:24:49.251533+0000","last_unstale":"2024-09-22T03:25:23.031701+0000","last_undegraded":"2024-09-22T03:25:23.031701+0000","last_fullsized":"2024-09-22T03:25:23.031701+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64487424,"num_objects":984,"num_object_clones":0,"num_object_copies":2952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":62976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":270,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-22T03:24:50.123717+0000","last_change":"2024-09-22T03:24:03.156391+0000","last_active":"2024-09-22T03:24:50.123717+0000","last_peered":"2024-09-22T03:24:50.123717+0000","last_clean":"2024-09-22T03:24:50.123717+0000","last_became_active":"2024-09-22T03:24:02.324247+0000","last_became_peered":"2024-09-22T03:24:02.324247+0000","last_unstale":"2024-09-22T03:24:50.123717+0000","last_undegraded":"2024-09-22T03:24:50.123717+0000","last_fullsized":"2024-09-22T03:24:50.123717+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":1009123328,"num_objects":15398,"num_object_clones":0,"num_object_copies":46194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15398,"num_write_kb":985472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187674624,"data_stored":3002793984,"data_compressed":142585395,"data_compressed_allocated":187674624,"data_compressed_original":3002793984,"omap_allocated":0,"internal_metadata":0},"log_size":15382,"ondisk_log_size":15382,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599092,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":120472,"kb_used_data":43848,"kb_used_omap":0,"kb_used_meta":76608,"kb_avail":94251368,"statfs":{"total":96636764160,"available":96513400832,"internally_reserved":0,"allocated":44900352,"data_stored":695865530,"data_compressed":33014452,"data_compressed_allocated":43454464,"data_compressed_original":695271424,"omap_allocated":0,"internal_metadata":78446592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":172,"seq":738734374946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79740,"kb_used_data":43820,"kb_used_omap":0,"kb_used_meta":35904,"kb_avail":94292100,"statfs":{"total":96636764160,"available":96555110400,"internally_reserved":0,"allocated":44871680,"data_stored":695014106,"data_compressed":32975968,"data_compressed_allocated":43405312,"data_compressed_original":694403072,"omap_allocated":0,"internal_metadata":36765696},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800144,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116256,"kb_used_data":50640,"kb_used_omap":0,"kb_used_meta":65600,"kb_avail":94255584,"statfs":{"total":96636764160,"available":96517718016,"internally_reserved":0,"allocated":51855360,"data_stored":806752986,"data_compressed":38281757,"data_compressed_allocated":50388992,"data_compressed_original":806141952,"omap_allocated":0,"internal_metadata":67174400},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":97,"seq":416611828052,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121872,"kb_used_data":47616,"kb_used_omap":0,"kb_used_meta":74240,"kb_avail":94249968,"statfs":{"total":96636764160,"available":96511967232,"internally_reserved":0,"allocated":48758784,"data_stored":757273287,"data_compressed":35932206,"data_compressed_allocated":47296512,"data_compressed_original":756662272,"omap_allocated":0,"internal_metadata":76021760},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":47878144,"data_stored":766050304,"data_compressed":36375258,"data_compressed_allocated":47878144,"data_compressed_original":766050304,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51367936,"data_stored":821886976,"data_compressed":39026673,"data_compressed_allocated":51367936,"data_compressed_original":821886976,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":44228608,"data_stored":707657728,"data_compressed":33602628,"data_compressed_allocated":44228608,"data_compressed_original":707657728,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":44199936,"data_stored":707198976,"data_compressed":33580836,"data_compressed_allocated":44199936,"data_compressed_original":707198976,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:25.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 64 16 16510 16494 16.1058 19.75 0.060298 0.0620627 2024-09-22T03:25:26.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 65 16 16862 16846 16.1965 22 0.0284205 0.0617141 2024-09-22T03:25:27.291 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:27.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:27.444+0000 7ffaf090e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:27.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:27.450+0000 7ffaf090e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:27.543 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 66 16 17200 17184 16.2711 21.125 0.0413966 0.0614273 2024-09-22T03:25:28.544 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 67 16 17525 17509 16.3314 20.3125 0.0588347 0.0612044 2024-09-22T03:25:29.544 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 68 16 17852 17836 16.3918 20.4375 0.0357184 0.0609693 2024-09-22T03:25:29.601 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:29.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:29.735+0000 7efc2d757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:29.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:29.743+0000 7efc2d757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:30.073 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 3, 1, 2] 2024-09-22T03:25:30.073 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:30.073 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 3, 1, 2] 2024-09-22T03:25:30.074 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T03:25:30.074 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:25:30.544 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 69 16 18023 18007 16.3091 10.6875 0.0169075 0.0608931 2024-09-22T03:25:31.544 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 70 16 18023 18007 16.0761 0 - 0.0608931 2024-09-22T03:25:31.983 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:32.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:32.096+0000 7fa0aab97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:32.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:25:32.101+0000 7fa0aab97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:32.544 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 71 16 18023 18007 15.8497 0 - 0.0608931 2024-09-22T03:25:33.544 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 72 16 18023 18007 15.6295 0 - 0.0608931 2024-09-22T03:25:34.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 73 16 18023 18007 15.4154 0 - 0.0608931 2024-09-22T03:25:35.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-22T03:25:35.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 74 16 18023 18007 15.2071 0 - 0.0608931 2024-09-22T03:25:36.174 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T03:25:36.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 75 16 18023 18007 15.0043 0 - 0.0608931 2024-09-22T03:25:37.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 76 16 18023 18007 14.8069 0 - 0.0608931 2024-09-22T03:25:38.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 77 16 18023 18007 14.6146 0 - 0.0608931 2024-09-22T03:25:39.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 78 16 18023 18007 14.4272 0 - 0.0608931 2024-09-22T03:25:40.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 79 16 18023 18007 14.2446 0 - 0.0608931 2024-09-22T03:25:40.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-22T03:25:41.175 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [3, 1, 2] 2024-09-22T03:25:41.176 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:41.176 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [3, 1, 2] 2024-09-22T03:25:41.176 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:25:41.176 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:25:41.545 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:25:41.545288+0000 min lat: 0.00860919 max lat: 19.7689 avg lat: 0.0608931 2024-09-22T03:25:41.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:25:41.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 80 16 18023 18007 14.0665 0 - 0.0608931 2024-09-22T03:25:42.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 81 16 18023 18007 13.8928 0 - 0.0608931 2024-09-22T03:25:43.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 82 16 18023 18007 13.7234 0 - 0.0608931 2024-09-22T03:25:44.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 83 16 18023 18007 13.5581 0 - 0.0608931 2024-09-22T03:25:45.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 84 16 18023 18007 13.3967 0 - 0.0608931 2024-09-22T03:25:46.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:25:46.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 85 16 18023 18007 13.239 0 - 0.0608931 2024-09-22T03:25:47.277 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:25:47.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 86 16 18023 18007 13.0851 0 - 0.0608931 2024-09-22T03:25:48.546 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 87 16 18023 18007 12.9347 0 - 0.0608931 2024-09-22T03:25:49.547 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 88 16 18023 18007 12.7877 0 - 0.0608931 2024-09-22T03:25:50.547 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 89 16 18023 18007 12.644 0 - 0.0608931 2024-09-22T03:25:51.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:25:51.547 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 90 16 18023 18007 12.5035 0 - 0.0608931 2024-09-22T03:25:52.278 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 2] live_osds: [3, 1] 2024-09-22T03:25:52.278 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:25:52.278 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-22T03:25:52.279 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:25:52.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:25:52.282 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:25:52.282 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:25:52.393 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:25:52.396 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:52.396 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:25:52.547 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 91 16 18023 18007 12.3661 0 - 0.0608931 2024-09-22T03:25:53.547 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 92 16 18023 18007 12.2317 0 - 0.0608931 2024-09-22T03:25:54.547 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 93 16 18023 18007 12.1002 0 - 0.0608931 2024-09-22T03:25:55.122 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:25:55.548 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 94 16 18023 18007 11.9714 0 - 0.0608931 2024-09-22T03:25:56.548 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 95 8 18023 18015 11.8507 0.0192308 26.1285 0.0724739 2024-09-22T03:25:57.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:25:57.494 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:25:57.496 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:57.496 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:25:57.548 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 96 8 18023 18015 11.7272 0 - 0.0724739 2024-09-22T03:25:58.548 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 97 8 18023 18015 11.6063 0 - 0.0724739 2024-09-22T03:25:59.548 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 98 8 18023 18015 11.4879 0 - 0.0724739 2024-09-22T03:26:00.548 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 99 8 18023 18015 11.3718 0 - 0.0724739 2024-09-22T03:26:01.548 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:26:01.548236+0000 min lat: 0.00860919 max lat: 26.1532 avg lat: 0.0724739 2024-09-22T03:26:01.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:26:01.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 100 8 18023 18015 11.2581 0 - 0.0724739 2024-09-22T03:26:02.127 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:26:02.176 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:02.176 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:02.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:26:02.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 101 8 18023 18015 11.1466 0 - 0.0724739 2024-09-22T03:26:02.597 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:26:02.598 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [] 2024-09-22T03:26:02.598 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:26:03.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 102 8 18023 18015 11.0374 0 - 0.0724739 2024-09-22T03:26:04.177 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:04.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:04.291+0000 7f2926d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:04.296+0000 7f2926d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.301 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:04.343 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:04.344 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:04.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 103 7 18023 18016 10.9308 0.0078125 34.1223 0.0743639 2024-09-22T03:26:05.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 104 7 18023 18016 10.8257 0 - 0.0743639 2024-09-22T03:26:06.344 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:06.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:06.458+0000 7f0ecdc4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:06.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:06.463+0000 7f0ecdc4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:06.468 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:06.514 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:06.515 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:06.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 105 7 18023 18016 10.7226 0 - 0.0743639 2024-09-22T03:26:07.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 106 7 18023 18016 10.6214 0 - 0.0743639 2024-09-22T03:26:07.608 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [3, 1, 2] 2024-09-22T03:26:07.608 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:26:07.608 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:07.608 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:07.720+0000 7f3f392b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:07.728+0000 7f3f392b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:07.736 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:07.737 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:07.737 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:07.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:07.836+0000 7fe293397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:07.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:07.844+0000 7fe293397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:07.857 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:26:07.858 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:26:07.861 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:07.861 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:07.861 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:07.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:07.961+0000 7f1f6a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:07.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:07.969+0000 7f1f6a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:07.982 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:07.982 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:26:07.982 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:26:07.982 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:26:07.983 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:26:07.985 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:07.986 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:07.986 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:08.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:08.085+0000 7f375a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:08.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:08.092+0000 7f375a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:26:08.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:26:08.108 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:26:08.110 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:08.110 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:08.516 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:08.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 107 7 18023 18016 10.5222 0 - 0.0743639 2024-09-22T03:26:08.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:08.616+0000 7f223d317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:08.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:08.621+0000 7f223d317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:08.626 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:08.671 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:08.672 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:09.549 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 108 7 18023 18016 10.4247 0 - 0.0743639 2024-09-22T03:26:10.550 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 109 7 18023 18016 10.3291 0 - 0.0743639 2024-09-22T03:26:10.673 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:10.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:10.785+0000 7fa88eece640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:10.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:10.790+0000 7fa88eece640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:10.795 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:10.842 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:10.843 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:11.550 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 110 7 18023 18016 10.2352 0 - 0.0743639 2024-09-22T03:26:12.550 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 111 7 18023 18016 10.143 0 - 0.0743639 2024-09-22T03:26:12.843 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:12.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:12.955+0000 7f3d80858640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:12.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:12.960+0000 7f3d80858640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:12.966 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:13.012 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:13.012 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:13.111 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [3, 1, 2] 2024-09-22T03:26:13.111 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:26:13.111 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:13.111 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:13.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.211+0000 7f6f9b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.218+0000 7f6f9b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.223 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:13.226 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:13.226 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:13.227 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:13.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.326+0000 7fe264a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.333+0000 7fe264a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:26:13.347 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:26:13.348 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:26:13.348 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:26:13.348 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:26:13.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:13.350 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:13.350 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:13.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.450+0000 7f809210e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.457+0000 7f809210e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:26:13.472 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:26:13.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:13.475 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:13.475 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:13.550 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 112 7 18023 18016 10.0524 0 - 0.0743639 2024-09-22T03:26:13.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.574+0000 7fb789b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:13.581+0000 7fb789b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.595 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:26:13.596 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:26:13.599 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:13.599 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:14.550 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 113 7 18023 18016 9.96345 0 - 0.0743639 2024-09-22T03:26:15.013 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:15.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:15.126+0000 7f27dad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:15.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:15.131+0000 7f27dad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:15.137 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:15.183 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:15.183 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:15.550 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 114 7 18023 18016 9.87605 0 - 0.0743639 2024-09-22T03:26:16.550 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 115 7 18023 18016 9.79017 0 - 0.0743639 2024-09-22T03:26:17.184 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:17.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:17.297+0000 7f421c197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:17.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:17.302+0000 7f421c197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:17.307 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:17.353 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:17.354 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total time run: 115.044 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total writes made: 18023 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Write size: 65536 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object size: 65536 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Bandwidth (MB/sec): 9.79133 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Bandwidth: 11.419 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max bandwidth (MB/sec): 26.875 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min bandwidth (MB/sec): 0 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average IOPS: 156 2024-09-22T03:26:17.551 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev IOPS: 182.712 2024-09-22T03:26:17.552 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max IOPS: 430 2024-09-22T03:26:17.552 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min IOPS: 0 2024-09-22T03:26:17.552 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average Latency(s): 0.0923951 2024-09-22T03:26:17.552 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Latency(s): 1.24322 2024-09-22T03:26:17.552 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max latency(s): 46.5168 2024-09-22T03:26:17.552 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min latency(s): 0.00860919 2024-09-22T03:26:17.566 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:26:18.600 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [3, 1, 2] 2024-09-22T03:26:18.600 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:26:18.600 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [3, 1, 2] 2024-09-22T03:26:18.600 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:26:18.600 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:26:19.354 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:19.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:19.459+0000 7f3967d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:19.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:19.463+0000 7f3967d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:19.468 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:26:24.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:26:24.701 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:26:29.701 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 2] live_osds: [3, 1] 2024-09-22T03:26:29.702 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-22T03:26:29.702 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:26:29.702 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:26:29.706 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:26:29.706 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:26:29.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:26:29.817 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:26:29.820 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:29.820 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:26:34.821 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:26:34.917 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:26:34.920 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:34.920 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:26:39.921 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:26:40.017 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:26:40.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:40.020 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:26:45.022 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:26:45.143 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:26:45.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:45.147 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:26:49.472 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:26:49.494 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:49.495 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:50.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 15, 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.173587+0000" 2024-09-22T03:26:50.254 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.173627+0000" 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.185029+0000" 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.255 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.185215+0000" 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.287662+0000" 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.256 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.288163+0000" 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.257 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.303383+0000" 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.258 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:48.199713+0000" 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.259 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:48.207145+0000" 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:48.209031+0000" 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.260 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.266949+0000" 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.261 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.262 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.263 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.264 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.265 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.266 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 16, 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.173950+0000" 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.173967+0000" 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.267 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.185330+0000" 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.268 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.288388+0000" 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.288726+0000" 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.270 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.317072+0000" 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:48.199901+0000" 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.271 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:48.211168+0000" 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.272 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.267732+0000" 2024-09-22T03:26:50.273 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.274 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.275 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.276 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.276 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.276 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.276 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.276 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.277 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.277 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.277 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.277 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.278 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.278 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.278 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.278 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.278 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.279 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.279 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.279 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.279 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.279 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.280 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.280 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.280 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.280 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.281 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.281 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.281 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.281 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.281 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.282 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.282 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.282 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.282 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.283 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.283 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.283 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.283 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.283 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.283 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.284 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.284 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.284 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 17, 2024-09-22T03:26:50.284 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.284 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.284 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.174161+0000" 2024-09-22T03:26:50.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.285 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.285 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.285 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.174166+0000" 2024-09-22T03:26:50.285 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.286 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.286 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.286 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.286 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.286 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.288845+0000" 2024-09-22T03:26:50.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.287 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.287 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.287 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.287 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.288 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.288 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.288 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.288 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.288 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.289249+0000" 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.289 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.290 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.290 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.290 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:46.289878+0000" 2024-09-22T03:26:50.290 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.291 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.351731+0000" 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:48.200070+0000" 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.292 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.267657+0000" 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.293 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.303973+0000" 2024-09-22T03:26:50.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.295 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.318343+0000" 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.297 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.322704+0000" 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.298 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.350988+0000" 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.299 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 342, 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.409468+0000" 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.300 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.409475+0000" 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.410147+0000" 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.410232+0000" 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.302 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.410286+0000" 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.303 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.437532+0000" 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.410686+0000" 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.304 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.410819+0000" 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.305 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.306 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.411531+0000" 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.307 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.411963+0000" 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.309 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.419165+0000" 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.310 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.436505+0000" 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.311 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 345, 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.425971+0000" 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.312 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.425982+0000" 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.426579+0000" 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.313 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.426737+0000" 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.426845+0000" 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.315 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.448385+0000" 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.427207+0000" 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.427536+0000" 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.427722+0000" 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.318 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.319 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.320 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.427913+0000" 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.432030+0000" 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.322 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.447306+0000" 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.323 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 348, 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.324 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.442000+0000" 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.442008+0000" 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.325 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.442367+0000" 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.442618+0000" 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.442694+0000" 2024-09-22T03:26:50.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.328 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.470210+0000" 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.442911+0000" 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.329 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.443298+0000" 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.443538+0000" 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.332 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.333 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.444627+0000" 2024-09-22T03:26:50.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.335 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.449309+0000" 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.465700+0000" 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.337 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 351, 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.456236+0000" 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.456244+0000" 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.338 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.456332+0000" 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.339 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.456430+0000" 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.340 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.456503+0000" 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.477153+0000" 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.457507+0000" 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.342 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.457661+0000" 2024-09-22T03:26:50.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.457786+0000" 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.345 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.346 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.457973+0000" 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.347 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.348 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.462606+0000" 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.349 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.476793+0000" 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.350 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 337, 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.351 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.391718+0000" 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.391725+0000" 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.393972+0000" 2024-09-22T03:26:50.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.353 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.394412+0000" 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.394816+0000" 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.355 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.418588+0000" 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.395020+0000" 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.395137+0000" 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.357 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.358 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.395223+0000" 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.359 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.360 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.395355+0000" 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.396121+0000" 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.362 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.363 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.418309+0000" 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.364 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 354, 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.473499+0000" 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.473507+0000" 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.365 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.473607+0000" 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.366 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.474933+0000" 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.475384+0000" 2024-09-22T03:26:50.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.368 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.498144+0000" 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.476104+0000" 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.476287+0000" 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.370 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.371 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.476460+0000" 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.372 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.476718+0000" 2024-09-22T03:26:50.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.477833+0000" 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.376 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.497884+0000" 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.377 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 359, 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.500979+0000" 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.378 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.500991+0000" 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.501203+0000" 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.380 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.501283+0000" 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.381 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.501445+0000" 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.382 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.518701+0000" 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.501667+0000" 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.383 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.501777+0000" 2024-09-22T03:26:50.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.385 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.501889+0000" 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.387 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.502032+0000" 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.388 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.504304+0000" 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.389 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.518267+0000" 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.390 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:26:50.391 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 362, 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.506429+0000" 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.506440+0000" 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.392 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.506510+0000" 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:26:50.393 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.506618+0000" 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.394 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.507883+0000" 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.395 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.527739+0000" 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.508104+0000" 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.396 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.508204+0000" 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.508306+0000" 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.398 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.399 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.400 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.508448+0000" 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.401 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.509403+0000" 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:26:49.527477+0000" 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.403 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:26:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:26:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:26:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:26:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:26:50.404 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:26:50.405 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:26:50.405 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:26:50.406 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-22T03:26:50.406 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:26:50.407 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:26:50.409 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:26:50.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:26:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:26:50.514 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:50.514 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:26:51.496 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:51.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:51.621+0000 7fd0ea4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:51.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:51.626+0000 7fd0ea4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:51.754 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:26:51.779 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:51.780 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:53.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:53.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:53.929+0000 7f7e92957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:53.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:53.935+0000 7f7e92957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:53.993 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:26:54.043 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:54.044 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:55.516 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:26:55.650 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:26:55.654 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:55.654 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:26:56.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:56.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:56.190+0000 7f2a8bcd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:56.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:56.198+0000 7f2a8bcd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:56.382 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:26:56.474 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:56.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:26:58.476 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:58.611+0000 7fe23670e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:58.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:26:58.617+0000 7fe23670e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:26:58.835 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:26:58.835 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:27:00.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:27:00.779 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:27:00.781 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:00.782 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-22T03:27:00.836 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:00.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:00.963+0000 7f1ca3997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:00.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:00.969+0000 7f1ca3997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:01.180 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-22T03:27:01.226 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:27:01.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:27:03.227 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:03.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:03.346+0000 7fdd1ded7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:03.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:03.351+0000 7fdd1ded7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:05.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:05.564+0000 7f2c06f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:05.573+0000 7f2c06f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.783 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T03:27:05.945 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:27:05.945 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [ 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 36, 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.218564+0000" 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.218603+0000" 2024-09-22T03:27:05.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.219199+0000" 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.947 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.219312+0000" 2024-09-22T03:27:05.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.219552+0000" 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.152213+0000" 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:05.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.220304+0000" 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.220723+0000" 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.220868+0000" 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:05.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.956 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 39, 2024-09-22T03:27:05.958 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.222835+0000" 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.222850+0000" 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.959 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.222992+0000" 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.960 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.223185+0000" 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.961 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.224196+0000" 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.155035+0000" 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.225438+0000" 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.226460+0000" 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.964 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.227186+0000" 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.970 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 43, 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.233514+0000" 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.971 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.233523+0000" 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.233913+0000" 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.972 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.234540+0000" 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.234749+0000" 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.974 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.155087+0000" 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.236065+0000" 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.975 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.236822+0000" 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.977 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.979 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.980 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 47, 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.240310+0000" 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.240318+0000" 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.983 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.241976+0000" 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.242791+0000" 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.985 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.243918+0000" 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.155133+0000" 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.245165+0000" 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.987 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.988 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.990 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:05.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.993 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 53, 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.252847+0000" 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.252853+0000" 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.253178+0000" 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.253265+0000" 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.253680+0000" 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.997 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.170370+0000" 2024-09-22T03:27:05.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:05.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.000 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.001 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.002 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:06.005 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:06.006 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery_sub", 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 395, 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": {} 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 37, 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.221060+0000" 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:06.007 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.221071+0000" 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.221571+0000" 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.008 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.221772+0000" 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.222420+0000" 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.010 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.177119+0000" 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.222653+0000" 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.222773+0000" 2024-09-22T03:27:06.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.222829+0000" 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.013 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.015 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.017 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:06.019 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 41, 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.227016+0000" 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.227024+0000" 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.020 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.227478+0000" 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.021 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.228111+0000" 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.229123+0000" 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.022 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.177172+0000" 2024-09-22T03:27:06.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.229364+0000" 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.024 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.229518+0000" 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.026 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.028 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:06.029 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.030 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 45, 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.237174+0000" 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.237182+0000" 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.032 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.237321+0000" 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.237916+0000" 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.034 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.239799+0000" 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:03.177222+0000" 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:27:02.240054+0000" 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.036 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:27:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.039 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.040 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.041 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:27:06.043 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery_sub", 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 33554779, 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": {} 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-22T03:27:06.044 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:27:06.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:27:06.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:06.199+0000 7fc8ddca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:06.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:06.214+0000 7fc8ddca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:07.153 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:27:07.170 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:27:07.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:07.320+0000 7f1a73e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:07.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:07.335+0000 7f1a73e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:07.806 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:07.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:07.955+0000 7f711cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:07.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:07.964+0000 7f711cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:08.158 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:27:08.179 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:27:08.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:08.334+0000 7fb84a02c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:08.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:08.347+0000 7fb84a02c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:09.163 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:27:09.185 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:27:09.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:09.331+0000 7f11f9d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:09.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:09.346+0000 7f11f9d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.131 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:27:10.147 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:27:10.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:10.242 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:10.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:10.291+0000 7eff29717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:10.306+0000 7eff29717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:10.404+0000 7f98a0d4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.413 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:10.412+0000 7f98a0d4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.598 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:10.598 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:10.611 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":668,"stamp":"2024-09-22T03:27:10.131645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":866976331,"num_objects":13225,"num_object_clones":0,"num_object_copies":39675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":22966,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22913,"ondisk_log_size":22913,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":396836,"kb_used_data":175264,"kb_used_omap":0,"kb_used_meta":221504,"kb_avail":377090524,"statfs":{"total":386547056640,"available":386140696576,"internally_reserved":0,"allocated":179470336,"data_stored":2772788273,"data_compressed":131549821,"data_compressed_allocated":173150208,"data_compressed_original":2770157568,"omap_allocated":0,"internal_metadata":226820096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-66781184,"num_objects":-1019,"num_object_clones":0,"num_object_copies":-3057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1019,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.701383"},"pg_stats":[{"pgid":"6.f","version":"188'1422","reported_seq":1668,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:06.514894+0000","last_change":"2024-09-22T03:27:04.483026+0000","last_active":"2024-09-22T03:27:06.514894+0000","last_peered":"2024-09-22T03:27:06.514894+0000","last_clean":"2024-09-22T03:25:30.095213+0000","last_became_active":"2024-09-22T03:27:04.310524+0000","last_became_peered":"2024-09-22T03:27:04.310524+0000","last_unstale":"2024-09-22T03:27:06.514894+0000","last_undegraded":"2024-09-22T03:27:06.514894+0000","last_fullsized":"2024-09-22T03:27:04.157647+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1422,"log_dups_size":0,"ondisk_log_size":1422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1423,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":837}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"188'1433","reported_seq":1546,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.041912+0000","last_change":"2024-09-22T03:27:04.444039+0000","last_active":"2024-09-22T03:27:07.041912+0000","last_peered":"2024-09-22T03:27:07.041912+0000","last_clean":"2024-09-22T03:25:29.946334+0000","last_became_active":"2024-09-22T03:27:04.436044+0000","last_became_peered":"2024-09-22T03:27:04.436044+0000","last_unstale":"2024-09-22T03:27:07.041912+0000","last_undegraded":"2024-09-22T03:27:04.216203+0000","last_fullsized":"2024-09-22T03:27:04.181958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1433,"log_dups_size":0,"ondisk_log_size":1433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54788096,"num_objects":836,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":836}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"188'1397","reported_seq":1516,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.010888+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:07.010888+0000","last_peered":"2024-09-22T03:27:07.010888+0000","last_clean":"2024-09-22T03:27:07.010888+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:07.010888+0000","last_undegraded":"2024-09-22T03:27:07.010888+0000","last_fullsized":"2024-09-22T03:27:07.010888+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1397,"log_dups_size":0,"ondisk_log_size":1397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51183640,"num_objects":782,"num_object_clones":0,"num_object_copies":2346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1398,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"189'1494","reported_seq":1575,"reported_epoch":189,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:08.070541+0000","last_change":"2024-09-22T03:27:04.339624+0000","last_active":"2024-09-22T03:27:08.070541+0000","last_peered":"2024-09-22T03:27:08.070541+0000","last_clean":"2024-09-22T03:26:47.209609+0000","last_became_active":"2024-09-22T03:27:04.335895+0000","last_became_peered":"2024-09-22T03:27:04.335895+0000","last_unstale":"2024-09-22T03:27:08.070541+0000","last_undegraded":"2024-09-22T03:27:04.201549+0000","last_fullsized":"2024-09-22T03:27:04.181884+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1494,"log_dups_size":0,"ondisk_log_size":1494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54722560,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":231,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1495,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":835}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"188'1451","reported_seq":1521,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.079110+0000","last_change":"2024-09-22T03:27:04.381432+0000","last_active":"2024-09-22T03:27:07.079110+0000","last_peered":"2024-09-22T03:27:07.079110+0000","last_clean":"2024-09-22T03:26:47.187237+0000","last_became_active":"2024-09-22T03:27:04.374674+0000","last_became_peered":"2024-09-22T03:27:04.374674+0000","last_unstale":"2024-09-22T03:27:07.079110+0000","last_undegraded":"2024-09-22T03:27:04.179504+0000","last_fullsized":"2024-09-22T03:27:04.158651+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1451,"log_dups_size":0,"ondisk_log_size":1451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54657024,"num_objects":834,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1452,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":834}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"188'1382","reported_seq":1486,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.496489+0000","last_change":"2024-09-22T03:27:04.465312+0000","last_active":"2024-09-22T03:27:06.496489+0000","last_peered":"2024-09-22T03:27:06.496489+0000","last_clean":"2024-09-22T03:25:30.104671+0000","last_became_active":"2024-09-22T03:27:04.461526+0000","last_became_peered":"2024-09-22T03:27:04.461526+0000","last_unstale":"2024-09-22T03:27:06.496489+0000","last_undegraded":"2024-09-22T03:27:04.273968+0000","last_fullsized":"2024-09-22T03:27:04.255541+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1382,"log_dups_size":0,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54329344,"num_objects":829,"num_object_clones":0,"num_object_copies":2487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1383,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":829}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"189'1420","reported_seq":1510,"reported_epoch":189,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.944657+0000","last_change":"2024-09-22T03:27:04.507740+0000","last_active":"2024-09-22T03:27:07.944657+0000","last_peered":"2024-09-22T03:27:07.944657+0000","last_clean":"2024-09-22T03:26:47.207978+0000","last_became_active":"2024-09-22T03:27:04.298212+0000","last_became_peered":"2024-09-22T03:27:04.298212+0000","last_unstale":"2024-09-22T03:27:07.944657+0000","last_undegraded":"2024-09-22T03:27:04.505944+0000","last_fullsized":"2024-09-22T03:27:04.158046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1420,"log_dups_size":0,"ondisk_log_size":1420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50659328,"num_objects":773,"num_object_clones":0,"num_object_copies":2319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1421,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":773}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"188'1329","reported_seq":1560,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:06.527192+0000","last_change":"2024-09-22T03:27:04.506655+0000","last_active":"2024-09-22T03:27:06.527192+0000","last_peered":"2024-09-22T03:27:06.527192+0000","last_clean":"2024-09-22T03:25:30.018775+0000","last_became_active":"2024-09-22T03:27:04.334971+0000","last_became_peered":"2024-09-22T03:27:04.334971+0000","last_unstale":"2024-09-22T03:27:06.527192+0000","last_undegraded":"2024-09-22T03:27:06.527192+0000","last_fullsized":"2024-09-22T03:27:04.179960+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53411840,"num_objects":815,"num_object_clones":0,"num_object_copies":2445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1329,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"188'1350","reported_seq":1438,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:06.430608+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:06.430608+0000","last_peered":"2024-09-22T03:27:06.430608+0000","last_clean":"2024-09-22T03:27:06.430608+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:06.430608+0000","last_undegraded":"2024-09-22T03:27:06.430608+0000","last_fullsized":"2024-09-22T03:27:06.430608+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1350,"log_dups_size":0,"ondisk_log_size":1350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53411840,"num_objects":815,"num_object_clones":0,"num_object_copies":2445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1351,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":340,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.178069+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:07.178069+0000","last_peered":"2024-09-22T03:27:07.178069+0000","last_clean":"2024-09-22T03:27:07.178069+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:07.178069+0000","last_undegraded":"2024-09-22T03:27:07.178069+0000","last_fullsized":"2024-09-22T03:27:07.178069+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"189'1479","reported_seq":1612,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:08.092239+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:08.092239+0000","last_peered":"2024-09-22T03:27:08.092239+0000","last_clean":"2024-09-22T03:27:08.092239+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:08.092239+0000","last_undegraded":"2024-09-22T03:27:08.092239+0000","last_fullsized":"2024-09-22T03:27:08.092239+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1479,"log_dups_size":0,"ondisk_log_size":1479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55312384,"num_objects":844,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":443,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.183329+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:05.183329+0000","last_peered":"2024-09-22T03:27:05.183329+0000","last_clean":"2024-09-22T03:27:05.183329+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:05.183329+0000","last_undegraded":"2024-09-22T03:27:05.183329+0000","last_fullsized":"2024-09-22T03:27:05.183329+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"188'1418","reported_seq":1540,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.485324+0000","last_change":"2024-09-22T03:27:04.479132+0000","last_active":"2024-09-22T03:27:06.485324+0000","last_peered":"2024-09-22T03:27:06.485324+0000","last_clean":"2024-09-22T03:25:30.080890+0000","last_became_active":"2024-09-22T03:27:04.474157+0000","last_became_peered":"2024-09-22T03:27:04.474157+0000","last_unstale":"2024-09-22T03:27:06.485324+0000","last_undegraded":"2024-09-22T03:27:04.292930+0000","last_fullsized":"2024-09-22T03:27:04.261632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55771136,"num_objects":851,"num_object_clones":0,"num_object_copies":2553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1419,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":851}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":462,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.176177+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:07.176177+0000","last_peered":"2024-09-22T03:27:07.176177+0000","last_clean":"2024-09-22T03:27:07.176177+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:07.176177+0000","last_undegraded":"2024-09-22T03:27:07.176177+0000","last_fullsized":"2024-09-22T03:27:07.176177+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"189'1482","reported_seq":1621,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:08.098832+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:08.098832+0000","last_peered":"2024-09-22T03:27:08.098832+0000","last_clean":"2024-09-22T03:27:08.098832+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:08.098832+0000","last_undegraded":"2024-09-22T03:27:08.098832+0000","last_fullsized":"2024-09-22T03:27:08.098832+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1482,"log_dups_size":0,"ondisk_log_size":1482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52625408,"num_objects":803,"num_object_clones":0,"num_object_copies":2409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1483,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":426,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.178657+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:07.178657+0000","last_peered":"2024-09-22T03:27:07.178657+0000","last_clean":"2024-09-22T03:27:07.178657+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:07.178657+0000","last_undegraded":"2024-09-22T03:27:07.178657+0000","last_fullsized":"2024-09-22T03:27:07.178657+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":561,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.185528+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:05.185528+0000","last_peered":"2024-09-22T03:27:05.185528+0000","last_clean":"2024-09-22T03:27:05.185528+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:05.185528+0000","last_undegraded":"2024-09-22T03:27:05.185528+0000","last_fullsized":"2024-09-22T03:27:05.185528+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"188'1409","reported_seq":1489,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:06.505623+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:06.505623+0000","last_peered":"2024-09-22T03:27:06.505623+0000","last_clean":"2024-09-22T03:27:06.505623+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:06.505623+0000","last_undegraded":"2024-09-22T03:27:06.505623+0000","last_fullsized":"2024-09-22T03:27:06.505623+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492032,"num_objects":862,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1410,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":236,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.170849+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:05.170849+0000","last_peered":"2024-09-22T03:27:05.170849+0000","last_clean":"2024-09-22T03:27:05.170849+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:05.170849+0000","last_undegraded":"2024-09-22T03:27:05.170849+0000","last_fullsized":"2024-09-22T03:27:05.170849+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"188'1442","reported_seq":1559,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.073631+0000","last_change":"2024-09-22T03:27:04.479082+0000","last_active":"2024-09-22T03:27:07.073631+0000","last_peered":"2024-09-22T03:27:07.073631+0000","last_clean":"2024-09-22T03:25:30.070791+0000","last_became_active":"2024-09-22T03:27:04.472230+0000","last_became_peered":"2024-09-22T03:27:04.472230+0000","last_unstale":"2024-09-22T03:27:07.073631+0000","last_undegraded":"2024-09-22T03:27:04.203862+0000","last_fullsized":"2024-09-22T03:27:04.183417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1442,"log_dups_size":0,"ondisk_log_size":1442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51707904,"num_objects":789,"num_object_clones":0,"num_object_copies":2367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1443,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":789}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":359,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.173395+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:07.173395+0000","last_peered":"2024-09-22T03:27:07.173395+0000","last_clean":"2024-09-22T03:27:07.173395+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:07.173395+0000","last_undegraded":"2024-09-22T03:27:07.173395+0000","last_fullsized":"2024-09-22T03:27:07.173395+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"188'1470","reported_seq":1842,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:07.106849+0000","last_change":"2024-09-22T03:27:04.528690+0000","last_active":"2024-09-22T03:27:07.106849+0000","last_peered":"2024-09-22T03:27:07.106849+0000","last_clean":"2024-09-22T03:25:30.106965+0000","last_became_active":"2024-09-22T03:27:04.461948+0000","last_became_peered":"2024-09-22T03:27:04.461948+0000","last_unstale":"2024-09-22T03:27:07.106849+0000","last_undegraded":"2024-09-22T03:27:07.106849+0000","last_fullsized":"2024-09-22T03:27:04.205110+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1470,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":858}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":366,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.171767+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:05.171767+0000","last_peered":"2024-09-22T03:27:05.171767+0000","last_clean":"2024-09-22T03:27:05.171767+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:05.171767+0000","last_undegraded":"2024-09-22T03:27:05.171767+0000","last_fullsized":"2024-09-22T03:27:05.171767+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"188'1435","reported_seq":1546,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.204877+0000","last_change":"2024-09-22T03:27:04.476260+0000","last_active":"2024-09-22T03:27:06.204877+0000","last_peered":"2024-09-22T03:27:06.204877+0000","last_clean":"2024-09-22T03:25:30.049680+0000","last_became_active":"2024-09-22T03:27:04.473583+0000","last_became_peered":"2024-09-22T03:27:04.473583+0000","last_unstale":"2024-09-22T03:27:06.204877+0000","last_undegraded":"2024-09-22T03:27:04.294867+0000","last_fullsized":"2024-09-22T03:27:04.275186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":858}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":313,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.168434+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:05.168434+0000","last_peered":"2024-09-22T03:27:05.168434+0000","last_clean":"2024-09-22T03:27:05.168434+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:05.168434+0000","last_undegraded":"2024-09-22T03:27:05.168434+0000","last_fullsized":"2024-09-22T03:27:05.168434+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":866385944,"num_objects":13221,"num_object_clones":0,"num_object_copies":39663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":22827,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175841280,"data_stored":2813263944,"data_compressed":133588074,"data_compressed_allocated":175828992,"data_compressed_original":2813263872,"omap_allocated":0,"internal_metadata":0},"log_size":22813,"ondisk_log_size":22813,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":118528,"kb_used_data":37744,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":94253312,"statfs":{"total":96636764160,"available":96515391488,"internally_reserved":0,"allocated":38649856,"data_stored":593879557,"data_compressed":28169824,"data_compressed_allocated":37076992,"data_compressed_original":593231872,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949765,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":80308,"kb_used_data":46692,"kb_used_omap":0,"kb_used_meta":33600,"kb_avail":94291532,"statfs":{"total":96636764160,"available":96554528768,"internally_reserved":0,"allocated":47812608,"data_stored":740283909,"data_compressed":35123687,"data_compressed_allocated":46231552,"data_compressed_original":739622912,"omap_allocated":0,"internal_metadata":34406400},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102676,"kb_used_data":45376,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94269164,"statfs":{"total":96636764160,"available":96531623936,"internally_reserved":0,"allocated":46465024,"data_stored":718657053,"data_compressed":34097035,"data_compressed_allocated":44879872,"data_compressed_original":717996032,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95324,"kb_used_data":45452,"kb_used_omap":0,"kb_used_meta":49856,"kb_avail":94276516,"statfs":{"total":96636764160,"available":96539152384,"internally_reserved":0,"allocated":46542848,"data_stored":719967754,"data_compressed":34159275,"data_compressed_allocated":44961792,"data_compressed_original":719306752,"omap_allocated":0,"internal_metadata":51052544},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44621824,"data_stored":713883672,"data_compressed":33899015,"data_compressed_allocated":44617728,"data_compressed_original":713883648,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43954176,"data_stored":703201304,"data_compressed":33391759,"data_compressed_allocated":43950080,"data_compressed_original":703201280,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50831360,"data_stored":813301760,"data_compressed":38619172,"data_compressed_allocated":50831360,"data_compressed_original":813301760,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36433920,"data_stored":582877208,"data_compressed":27678128,"data_compressed_allocated":36429824,"data_compressed_original":582877184,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:10.613 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:10.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:10.722+0000 7fdd55c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:10.729+0000 7fdd55c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.914 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:10.914 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:10.926 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":668,"stamp":"2024-09-22T03:27:10.131645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":866976331,"num_objects":13225,"num_object_clones":0,"num_object_copies":39675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":22966,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22913,"ondisk_log_size":22913,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":396836,"kb_used_data":175264,"kb_used_omap":0,"kb_used_meta":221504,"kb_avail":377090524,"statfs":{"total":386547056640,"available":386140696576,"internally_reserved":0,"allocated":179470336,"data_stored":2772788273,"data_compressed":131549821,"data_compressed_allocated":173150208,"data_compressed_original":2770157568,"omap_allocated":0,"internal_metadata":226820096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-66781184,"num_objects":-1019,"num_object_clones":0,"num_object_copies":-3057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1019,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.701383"},"pg_stats":[{"pgid":"6.f","version":"188'1422","reported_seq":1668,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:06.514894+0000","last_change":"2024-09-22T03:27:04.483026+0000","last_active":"2024-09-22T03:27:06.514894+0000","last_peered":"2024-09-22T03:27:06.514894+0000","last_clean":"2024-09-22T03:25:30.095213+0000","last_became_active":"2024-09-22T03:27:04.310524+0000","last_became_peered":"2024-09-22T03:27:04.310524+0000","last_unstale":"2024-09-22T03:27:06.514894+0000","last_undegraded":"2024-09-22T03:27:06.514894+0000","last_fullsized":"2024-09-22T03:27:04.157647+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1422,"log_dups_size":0,"ondisk_log_size":1422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1423,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":837}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"188'1433","reported_seq":1546,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.041912+0000","last_change":"2024-09-22T03:27:04.444039+0000","last_active":"2024-09-22T03:27:07.041912+0000","last_peered":"2024-09-22T03:27:07.041912+0000","last_clean":"2024-09-22T03:25:29.946334+0000","last_became_active":"2024-09-22T03:27:04.436044+0000","last_became_peered":"2024-09-22T03:27:04.436044+0000","last_unstale":"2024-09-22T03:27:07.041912+0000","last_undegraded":"2024-09-22T03:27:04.216203+0000","last_fullsized":"2024-09-22T03:27:04.181958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1433,"log_dups_size":0,"ondisk_log_size":1433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54788096,"num_objects":836,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":836}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"188'1397","reported_seq":1516,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.010888+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:07.010888+0000","last_peered":"2024-09-22T03:27:07.010888+0000","last_clean":"2024-09-22T03:27:07.010888+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:07.010888+0000","last_undegraded":"2024-09-22T03:27:07.010888+0000","last_fullsized":"2024-09-22T03:27:07.010888+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1397,"log_dups_size":0,"ondisk_log_size":1397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51183640,"num_objects":782,"num_object_clones":0,"num_object_copies":2346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1398,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"189'1494","reported_seq":1575,"reported_epoch":189,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:08.070541+0000","last_change":"2024-09-22T03:27:04.339624+0000","last_active":"2024-09-22T03:27:08.070541+0000","last_peered":"2024-09-22T03:27:08.070541+0000","last_clean":"2024-09-22T03:26:47.209609+0000","last_became_active":"2024-09-22T03:27:04.335895+0000","last_became_peered":"2024-09-22T03:27:04.335895+0000","last_unstale":"2024-09-22T03:27:08.070541+0000","last_undegraded":"2024-09-22T03:27:04.201549+0000","last_fullsized":"2024-09-22T03:27:04.181884+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1494,"log_dups_size":0,"ondisk_log_size":1494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54722560,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":231,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1495,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":835}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"188'1451","reported_seq":1521,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.079110+0000","last_change":"2024-09-22T03:27:04.381432+0000","last_active":"2024-09-22T03:27:07.079110+0000","last_peered":"2024-09-22T03:27:07.079110+0000","last_clean":"2024-09-22T03:26:47.187237+0000","last_became_active":"2024-09-22T03:27:04.374674+0000","last_became_peered":"2024-09-22T03:27:04.374674+0000","last_unstale":"2024-09-22T03:27:07.079110+0000","last_undegraded":"2024-09-22T03:27:04.179504+0000","last_fullsized":"2024-09-22T03:27:04.158651+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1451,"log_dups_size":0,"ondisk_log_size":1451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54657024,"num_objects":834,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1452,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":834}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"188'1382","reported_seq":1486,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.496489+0000","last_change":"2024-09-22T03:27:04.465312+0000","last_active":"2024-09-22T03:27:06.496489+0000","last_peered":"2024-09-22T03:27:06.496489+0000","last_clean":"2024-09-22T03:25:30.104671+0000","last_became_active":"2024-09-22T03:27:04.461526+0000","last_became_peered":"2024-09-22T03:27:04.461526+0000","last_unstale":"2024-09-22T03:27:06.496489+0000","last_undegraded":"2024-09-22T03:27:04.273968+0000","last_fullsized":"2024-09-22T03:27:04.255541+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1382,"log_dups_size":0,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54329344,"num_objects":829,"num_object_clones":0,"num_object_copies":2487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1383,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":829}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"189'1420","reported_seq":1510,"reported_epoch":189,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.944657+0000","last_change":"2024-09-22T03:27:04.507740+0000","last_active":"2024-09-22T03:27:07.944657+0000","last_peered":"2024-09-22T03:27:07.944657+0000","last_clean":"2024-09-22T03:26:47.207978+0000","last_became_active":"2024-09-22T03:27:04.298212+0000","last_became_peered":"2024-09-22T03:27:04.298212+0000","last_unstale":"2024-09-22T03:27:07.944657+0000","last_undegraded":"2024-09-22T03:27:04.505944+0000","last_fullsized":"2024-09-22T03:27:04.158046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1420,"log_dups_size":0,"ondisk_log_size":1420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50659328,"num_objects":773,"num_object_clones":0,"num_object_copies":2319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1421,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":773}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"188'1329","reported_seq":1560,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:06.527192+0000","last_change":"2024-09-22T03:27:04.506655+0000","last_active":"2024-09-22T03:27:06.527192+0000","last_peered":"2024-09-22T03:27:06.527192+0000","last_clean":"2024-09-22T03:25:30.018775+0000","last_became_active":"2024-09-22T03:27:04.334971+0000","last_became_peered":"2024-09-22T03:27:04.334971+0000","last_unstale":"2024-09-22T03:27:06.527192+0000","last_undegraded":"2024-09-22T03:27:06.527192+0000","last_fullsized":"2024-09-22T03:27:04.179960+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53411840,"num_objects":815,"num_object_clones":0,"num_object_copies":2445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1329,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"188'1350","reported_seq":1438,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:06.430608+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:06.430608+0000","last_peered":"2024-09-22T03:27:06.430608+0000","last_clean":"2024-09-22T03:27:06.430608+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:06.430608+0000","last_undegraded":"2024-09-22T03:27:06.430608+0000","last_fullsized":"2024-09-22T03:27:06.430608+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1350,"log_dups_size":0,"ondisk_log_size":1350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53411840,"num_objects":815,"num_object_clones":0,"num_object_copies":2445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1351,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":340,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.178069+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:07.178069+0000","last_peered":"2024-09-22T03:27:07.178069+0000","last_clean":"2024-09-22T03:27:07.178069+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:07.178069+0000","last_undegraded":"2024-09-22T03:27:07.178069+0000","last_fullsized":"2024-09-22T03:27:07.178069+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"189'1479","reported_seq":1612,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:08.092239+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:08.092239+0000","last_peered":"2024-09-22T03:27:08.092239+0000","last_clean":"2024-09-22T03:27:08.092239+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:08.092239+0000","last_undegraded":"2024-09-22T03:27:08.092239+0000","last_fullsized":"2024-09-22T03:27:08.092239+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1479,"log_dups_size":0,"ondisk_log_size":1479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55312384,"num_objects":844,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":443,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.183329+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:05.183329+0000","last_peered":"2024-09-22T03:27:05.183329+0000","last_clean":"2024-09-22T03:27:05.183329+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:05.183329+0000","last_undegraded":"2024-09-22T03:27:05.183329+0000","last_fullsized":"2024-09-22T03:27:05.183329+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"188'1418","reported_seq":1540,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.485324+0000","last_change":"2024-09-22T03:27:04.479132+0000","last_active":"2024-09-22T03:27:06.485324+0000","last_peered":"2024-09-22T03:27:06.485324+0000","last_clean":"2024-09-22T03:25:30.080890+0000","last_became_active":"2024-09-22T03:27:04.474157+0000","last_became_peered":"2024-09-22T03:27:04.474157+0000","last_unstale":"2024-09-22T03:27:06.485324+0000","last_undegraded":"2024-09-22T03:27:04.292930+0000","last_fullsized":"2024-09-22T03:27:04.261632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55771136,"num_objects":851,"num_object_clones":0,"num_object_copies":2553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1419,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":851}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":462,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.176177+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:07.176177+0000","last_peered":"2024-09-22T03:27:07.176177+0000","last_clean":"2024-09-22T03:27:07.176177+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:07.176177+0000","last_undegraded":"2024-09-22T03:27:07.176177+0000","last_fullsized":"2024-09-22T03:27:07.176177+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"189'1482","reported_seq":1621,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:08.098832+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:08.098832+0000","last_peered":"2024-09-22T03:27:08.098832+0000","last_clean":"2024-09-22T03:27:08.098832+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:08.098832+0000","last_undegraded":"2024-09-22T03:27:08.098832+0000","last_fullsized":"2024-09-22T03:27:08.098832+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1482,"log_dups_size":0,"ondisk_log_size":1482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52625408,"num_objects":803,"num_object_clones":0,"num_object_copies":2409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1483,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":426,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.178657+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:07.178657+0000","last_peered":"2024-09-22T03:27:07.178657+0000","last_clean":"2024-09-22T03:27:07.178657+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:07.178657+0000","last_undegraded":"2024-09-22T03:27:07.178657+0000","last_fullsized":"2024-09-22T03:27:07.178657+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":561,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.185528+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:05.185528+0000","last_peered":"2024-09-22T03:27:05.185528+0000","last_clean":"2024-09-22T03:27:05.185528+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:05.185528+0000","last_undegraded":"2024-09-22T03:27:05.185528+0000","last_fullsized":"2024-09-22T03:27:05.185528+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"188'1409","reported_seq":1489,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:06.505623+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:06.505623+0000","last_peered":"2024-09-22T03:27:06.505623+0000","last_clean":"2024-09-22T03:27:06.505623+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:06.505623+0000","last_undegraded":"2024-09-22T03:27:06.505623+0000","last_fullsized":"2024-09-22T03:27:06.505623+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492032,"num_objects":862,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1410,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":236,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.170849+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:05.170849+0000","last_peered":"2024-09-22T03:27:05.170849+0000","last_clean":"2024-09-22T03:27:05.170849+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:05.170849+0000","last_undegraded":"2024-09-22T03:27:05.170849+0000","last_fullsized":"2024-09-22T03:27:05.170849+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"188'1442","reported_seq":1559,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.073631+0000","last_change":"2024-09-22T03:27:04.479082+0000","last_active":"2024-09-22T03:27:07.073631+0000","last_peered":"2024-09-22T03:27:07.073631+0000","last_clean":"2024-09-22T03:25:30.070791+0000","last_became_active":"2024-09-22T03:27:04.472230+0000","last_became_peered":"2024-09-22T03:27:04.472230+0000","last_unstale":"2024-09-22T03:27:07.073631+0000","last_undegraded":"2024-09-22T03:27:04.203862+0000","last_fullsized":"2024-09-22T03:27:04.183417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1442,"log_dups_size":0,"ondisk_log_size":1442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51707904,"num_objects":789,"num_object_clones":0,"num_object_copies":2367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1443,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":789}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":359,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.173395+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:07.173395+0000","last_peered":"2024-09-22T03:27:07.173395+0000","last_clean":"2024-09-22T03:27:07.173395+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:07.173395+0000","last_undegraded":"2024-09-22T03:27:07.173395+0000","last_fullsized":"2024-09-22T03:27:07.173395+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"188'1470","reported_seq":1842,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:07.106849+0000","last_change":"2024-09-22T03:27:04.528690+0000","last_active":"2024-09-22T03:27:07.106849+0000","last_peered":"2024-09-22T03:27:07.106849+0000","last_clean":"2024-09-22T03:25:30.106965+0000","last_became_active":"2024-09-22T03:27:04.461948+0000","last_became_peered":"2024-09-22T03:27:04.461948+0000","last_unstale":"2024-09-22T03:27:07.106849+0000","last_undegraded":"2024-09-22T03:27:07.106849+0000","last_fullsized":"2024-09-22T03:27:04.205110+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1470,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":858}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":366,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.171767+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:05.171767+0000","last_peered":"2024-09-22T03:27:05.171767+0000","last_clean":"2024-09-22T03:27:05.171767+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:05.171767+0000","last_undegraded":"2024-09-22T03:27:05.171767+0000","last_fullsized":"2024-09-22T03:27:05.171767+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"188'1435","reported_seq":1546,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.204877+0000","last_change":"2024-09-22T03:27:04.476260+0000","last_active":"2024-09-22T03:27:06.204877+0000","last_peered":"2024-09-22T03:27:06.204877+0000","last_clean":"2024-09-22T03:25:30.049680+0000","last_became_active":"2024-09-22T03:27:04.473583+0000","last_became_peered":"2024-09-22T03:27:04.473583+0000","last_unstale":"2024-09-22T03:27:06.204877+0000","last_undegraded":"2024-09-22T03:27:04.294867+0000","last_fullsized":"2024-09-22T03:27:04.275186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":858}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":313,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.168434+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:05.168434+0000","last_peered":"2024-09-22T03:27:05.168434+0000","last_clean":"2024-09-22T03:27:05.168434+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:05.168434+0000","last_undegraded":"2024-09-22T03:27:05.168434+0000","last_fullsized":"2024-09-22T03:27:05.168434+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":866385944,"num_objects":13221,"num_object_clones":0,"num_object_copies":39663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":22827,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175841280,"data_stored":2813263944,"data_compressed":133588074,"data_compressed_allocated":175828992,"data_compressed_original":2813263872,"omap_allocated":0,"internal_metadata":0},"log_size":22813,"ondisk_log_size":22813,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":118528,"kb_used_data":37744,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":94253312,"statfs":{"total":96636764160,"available":96515391488,"internally_reserved":0,"allocated":38649856,"data_stored":593879557,"data_compressed":28169824,"data_compressed_allocated":37076992,"data_compressed_original":593231872,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949765,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":80308,"kb_used_data":46692,"kb_used_omap":0,"kb_used_meta":33600,"kb_avail":94291532,"statfs":{"total":96636764160,"available":96554528768,"internally_reserved":0,"allocated":47812608,"data_stored":740283909,"data_compressed":35123687,"data_compressed_allocated":46231552,"data_compressed_original":739622912,"omap_allocated":0,"internal_metadata":34406400},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102676,"kb_used_data":45376,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94269164,"statfs":{"total":96636764160,"available":96531623936,"internally_reserved":0,"allocated":46465024,"data_stored":718657053,"data_compressed":34097035,"data_compressed_allocated":44879872,"data_compressed_original":717996032,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95324,"kb_used_data":45452,"kb_used_omap":0,"kb_used_meta":49856,"kb_avail":94276516,"statfs":{"total":96636764160,"available":96539152384,"internally_reserved":0,"allocated":46542848,"data_stored":719967754,"data_compressed":34159275,"data_compressed_allocated":44961792,"data_compressed_original":719306752,"omap_allocated":0,"internal_metadata":51052544},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44621824,"data_stored":713883672,"data_compressed":33899015,"data_compressed_allocated":44617728,"data_compressed_original":713883648,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43954176,"data_stored":703201304,"data_compressed":33391759,"data_compressed_allocated":43950080,"data_compressed_original":703201280,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50831360,"data_stored":813301760,"data_compressed":38619172,"data_compressed_allocated":50831360,"data_compressed_original":813301760,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36433920,"data_stored":582877208,"data_compressed":27678128,"data_compressed_allocated":36429824,"data_compressed_original":582877184,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:10.928 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:27:11.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:11.030+0000 7ff8e1517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:11.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:11.036+0000 7ff8e1517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:11.427 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:11.427 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1823/39675 objects degraded (4.595%), 8 pgs degraded","count":8},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":976,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":193,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975622,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+remapped","count":3}],"num_pgs":25,"num_pools":3,"num_objects":13225,"data_bytes":866976331,"bytes_used":406360064,"bytes_avail":386140696576,"bytes_total":386547056640,"degraded_objects":1823,"degraded_total":39675,"degraded_ratio":0.045948330182734719,"recovering_objects_per_sec":65,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":280,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":236},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T03:27:01.429211+0000","services":{}},"progress_events":{}} 2024-09-22T03:27:11.445 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:27:11.445 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:11.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:11.565+0000 7f2bd126c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:11.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:11.574+0000 7f2bd126c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:11.758 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:11.759 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:11.771 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":670,"stamp":"2024-09-22T03:27:11.431320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":866976331,"num_objects":13225,"num_object_clones":0,"num_object_copies":39675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":22966,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22913,"ondisk_log_size":22913,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":396836,"kb_used_data":175264,"kb_used_omap":0,"kb_used_meta":221504,"kb_avail":377090524,"statfs":{"total":386547056640,"available":386140696576,"internally_reserved":0,"allocated":179470336,"data_stored":2772788273,"data_compressed":131549821,"data_compressed_allocated":173150208,"data_compressed_original":2770157568,"omap_allocated":0,"internal_metadata":226820096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18481152,"num_objects":-282,"num_object_clones":0,"num_object_copies":-846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000713"},"pg_stats":[{"pgid":"6.f","version":"188'1422","reported_seq":1668,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:06.514894+0000","last_change":"2024-09-22T03:27:04.483026+0000","last_active":"2024-09-22T03:27:06.514894+0000","last_peered":"2024-09-22T03:27:06.514894+0000","last_clean":"2024-09-22T03:25:30.095213+0000","last_became_active":"2024-09-22T03:27:04.310524+0000","last_became_peered":"2024-09-22T03:27:04.310524+0000","last_unstale":"2024-09-22T03:27:06.514894+0000","last_undegraded":"2024-09-22T03:27:06.514894+0000","last_fullsized":"2024-09-22T03:27:04.157647+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1422,"log_dups_size":0,"ondisk_log_size":1422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1423,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":837}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"188'1433","reported_seq":1546,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.041912+0000","last_change":"2024-09-22T03:27:04.444039+0000","last_active":"2024-09-22T03:27:07.041912+0000","last_peered":"2024-09-22T03:27:07.041912+0000","last_clean":"2024-09-22T03:25:29.946334+0000","last_became_active":"2024-09-22T03:27:04.436044+0000","last_became_peered":"2024-09-22T03:27:04.436044+0000","last_unstale":"2024-09-22T03:27:07.041912+0000","last_undegraded":"2024-09-22T03:27:04.216203+0000","last_fullsized":"2024-09-22T03:27:04.181958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1433,"log_dups_size":0,"ondisk_log_size":1433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54788096,"num_objects":836,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":836}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"188'1397","reported_seq":1516,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.010888+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:07.010888+0000","last_peered":"2024-09-22T03:27:07.010888+0000","last_clean":"2024-09-22T03:27:07.010888+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:07.010888+0000","last_undegraded":"2024-09-22T03:27:07.010888+0000","last_fullsized":"2024-09-22T03:27:07.010888+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1397,"log_dups_size":0,"ondisk_log_size":1397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51183640,"num_objects":782,"num_object_clones":0,"num_object_copies":2346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1398,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"189'1494","reported_seq":1575,"reported_epoch":189,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:08.070541+0000","last_change":"2024-09-22T03:27:04.339624+0000","last_active":"2024-09-22T03:27:08.070541+0000","last_peered":"2024-09-22T03:27:08.070541+0000","last_clean":"2024-09-22T03:26:47.209609+0000","last_became_active":"2024-09-22T03:27:04.335895+0000","last_became_peered":"2024-09-22T03:27:04.335895+0000","last_unstale":"2024-09-22T03:27:08.070541+0000","last_undegraded":"2024-09-22T03:27:04.201549+0000","last_fullsized":"2024-09-22T03:27:04.181884+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1494,"log_dups_size":0,"ondisk_log_size":1494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54722560,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":231,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1495,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":835}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"188'1451","reported_seq":1521,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.079110+0000","last_change":"2024-09-22T03:27:04.381432+0000","last_active":"2024-09-22T03:27:07.079110+0000","last_peered":"2024-09-22T03:27:07.079110+0000","last_clean":"2024-09-22T03:26:47.187237+0000","last_became_active":"2024-09-22T03:27:04.374674+0000","last_became_peered":"2024-09-22T03:27:04.374674+0000","last_unstale":"2024-09-22T03:27:07.079110+0000","last_undegraded":"2024-09-22T03:27:04.179504+0000","last_fullsized":"2024-09-22T03:27:04.158651+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1451,"log_dups_size":0,"ondisk_log_size":1451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54657024,"num_objects":834,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1452,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":834}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"188'1382","reported_seq":1486,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.496489+0000","last_change":"2024-09-22T03:27:04.465312+0000","last_active":"2024-09-22T03:27:06.496489+0000","last_peered":"2024-09-22T03:27:06.496489+0000","last_clean":"2024-09-22T03:25:30.104671+0000","last_became_active":"2024-09-22T03:27:04.461526+0000","last_became_peered":"2024-09-22T03:27:04.461526+0000","last_unstale":"2024-09-22T03:27:06.496489+0000","last_undegraded":"2024-09-22T03:27:04.273968+0000","last_fullsized":"2024-09-22T03:27:04.255541+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1382,"log_dups_size":0,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54329344,"num_objects":829,"num_object_clones":0,"num_object_copies":2487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1383,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":829}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"189'1420","reported_seq":1510,"reported_epoch":189,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.944657+0000","last_change":"2024-09-22T03:27:04.507740+0000","last_active":"2024-09-22T03:27:07.944657+0000","last_peered":"2024-09-22T03:27:07.944657+0000","last_clean":"2024-09-22T03:26:47.207978+0000","last_became_active":"2024-09-22T03:27:04.298212+0000","last_became_peered":"2024-09-22T03:27:04.298212+0000","last_unstale":"2024-09-22T03:27:07.944657+0000","last_undegraded":"2024-09-22T03:27:04.505944+0000","last_fullsized":"2024-09-22T03:27:04.158046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1420,"log_dups_size":0,"ondisk_log_size":1420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50659328,"num_objects":773,"num_object_clones":0,"num_object_copies":2319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1421,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":773}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"188'1329","reported_seq":1560,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:06.527192+0000","last_change":"2024-09-22T03:27:04.506655+0000","last_active":"2024-09-22T03:27:06.527192+0000","last_peered":"2024-09-22T03:27:06.527192+0000","last_clean":"2024-09-22T03:25:30.018775+0000","last_became_active":"2024-09-22T03:27:04.334971+0000","last_became_peered":"2024-09-22T03:27:04.334971+0000","last_unstale":"2024-09-22T03:27:06.527192+0000","last_undegraded":"2024-09-22T03:27:06.527192+0000","last_fullsized":"2024-09-22T03:27:04.179960+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53411840,"num_objects":815,"num_object_clones":0,"num_object_copies":2445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1329,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"188'1350","reported_seq":1438,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:06.430608+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:06.430608+0000","last_peered":"2024-09-22T03:27:06.430608+0000","last_clean":"2024-09-22T03:27:06.430608+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:06.430608+0000","last_undegraded":"2024-09-22T03:27:06.430608+0000","last_fullsized":"2024-09-22T03:27:06.430608+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1350,"log_dups_size":0,"ondisk_log_size":1350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53411840,"num_objects":815,"num_object_clones":0,"num_object_copies":2445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1351,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":340,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.178069+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:07.178069+0000","last_peered":"2024-09-22T03:27:07.178069+0000","last_clean":"2024-09-22T03:27:07.178069+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:07.178069+0000","last_undegraded":"2024-09-22T03:27:07.178069+0000","last_fullsized":"2024-09-22T03:27:07.178069+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"189'1479","reported_seq":1612,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:08.092239+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:08.092239+0000","last_peered":"2024-09-22T03:27:08.092239+0000","last_clean":"2024-09-22T03:27:08.092239+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:08.092239+0000","last_undegraded":"2024-09-22T03:27:08.092239+0000","last_fullsized":"2024-09-22T03:27:08.092239+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1479,"log_dups_size":0,"ondisk_log_size":1479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55312384,"num_objects":844,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":443,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.183329+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:05.183329+0000","last_peered":"2024-09-22T03:27:05.183329+0000","last_clean":"2024-09-22T03:27:05.183329+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:05.183329+0000","last_undegraded":"2024-09-22T03:27:05.183329+0000","last_fullsized":"2024-09-22T03:27:05.183329+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"188'1418","reported_seq":1540,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.485324+0000","last_change":"2024-09-22T03:27:04.479132+0000","last_active":"2024-09-22T03:27:06.485324+0000","last_peered":"2024-09-22T03:27:06.485324+0000","last_clean":"2024-09-22T03:25:30.080890+0000","last_became_active":"2024-09-22T03:27:04.474157+0000","last_became_peered":"2024-09-22T03:27:04.474157+0000","last_unstale":"2024-09-22T03:27:06.485324+0000","last_undegraded":"2024-09-22T03:27:04.292930+0000","last_fullsized":"2024-09-22T03:27:04.261632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55771136,"num_objects":851,"num_object_clones":0,"num_object_copies":2553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1419,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":851}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":462,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.176177+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:07.176177+0000","last_peered":"2024-09-22T03:27:07.176177+0000","last_clean":"2024-09-22T03:27:07.176177+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:07.176177+0000","last_undegraded":"2024-09-22T03:27:07.176177+0000","last_fullsized":"2024-09-22T03:27:07.176177+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"189'1482","reported_seq":1621,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:08.098832+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:08.098832+0000","last_peered":"2024-09-22T03:27:08.098832+0000","last_clean":"2024-09-22T03:27:08.098832+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:08.098832+0000","last_undegraded":"2024-09-22T03:27:08.098832+0000","last_fullsized":"2024-09-22T03:27:08.098832+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1482,"log_dups_size":0,"ondisk_log_size":1482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52625408,"num_objects":803,"num_object_clones":0,"num_object_copies":2409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1483,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":426,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.178657+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:07.178657+0000","last_peered":"2024-09-22T03:27:07.178657+0000","last_clean":"2024-09-22T03:27:07.178657+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:07.178657+0000","last_undegraded":"2024-09-22T03:27:07.178657+0000","last_fullsized":"2024-09-22T03:27:07.178657+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":561,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.185528+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:05.185528+0000","last_peered":"2024-09-22T03:27:05.185528+0000","last_clean":"2024-09-22T03:27:05.185528+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:05.185528+0000","last_undegraded":"2024-09-22T03:27:05.185528+0000","last_fullsized":"2024-09-22T03:27:05.185528+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"188'1409","reported_seq":1489,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:06.505623+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:06.505623+0000","last_peered":"2024-09-22T03:27:06.505623+0000","last_clean":"2024-09-22T03:27:06.505623+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:06.505623+0000","last_undegraded":"2024-09-22T03:27:06.505623+0000","last_fullsized":"2024-09-22T03:27:06.505623+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56492032,"num_objects":862,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1410,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":236,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.170849+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:05.170849+0000","last_peered":"2024-09-22T03:27:05.170849+0000","last_clean":"2024-09-22T03:27:05.170849+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:05.170849+0000","last_undegraded":"2024-09-22T03:27:05.170849+0000","last_fullsized":"2024-09-22T03:27:05.170849+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"188'1442","reported_seq":1559,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:07.073631+0000","last_change":"2024-09-22T03:27:04.479082+0000","last_active":"2024-09-22T03:27:07.073631+0000","last_peered":"2024-09-22T03:27:07.073631+0000","last_clean":"2024-09-22T03:25:30.070791+0000","last_became_active":"2024-09-22T03:27:04.472230+0000","last_became_peered":"2024-09-22T03:27:04.472230+0000","last_unstale":"2024-09-22T03:27:07.073631+0000","last_undegraded":"2024-09-22T03:27:04.203862+0000","last_fullsized":"2024-09-22T03:27:04.183417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1442,"log_dups_size":0,"ondisk_log_size":1442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51707904,"num_objects":789,"num_object_clones":0,"num_object_copies":2367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1443,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":789}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":359,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-22T03:27:07.173395+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:07.173395+0000","last_peered":"2024-09-22T03:27:07.173395+0000","last_clean":"2024-09-22T03:27:07.173395+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:07.173395+0000","last_undegraded":"2024-09-22T03:27:07.173395+0000","last_fullsized":"2024-09-22T03:27:07.173395+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"188'1470","reported_seq":1842,"reported_epoch":188,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:07.106849+0000","last_change":"2024-09-22T03:27:04.528690+0000","last_active":"2024-09-22T03:27:07.106849+0000","last_peered":"2024-09-22T03:27:07.106849+0000","last_clean":"2024-09-22T03:25:30.106965+0000","last_became_active":"2024-09-22T03:27:04.461948+0000","last_became_peered":"2024-09-22T03:27:04.461948+0000","last_unstale":"2024-09-22T03:27:07.106849+0000","last_undegraded":"2024-09-22T03:27:07.106849+0000","last_fullsized":"2024-09-22T03:27:04.205110+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1470,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":858}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":366,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.171767+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:05.171767+0000","last_peered":"2024-09-22T03:27:05.171767+0000","last_clean":"2024-09-22T03:27:05.171767+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:05.171767+0000","last_undegraded":"2024-09-22T03:27:05.171767+0000","last_fullsized":"2024-09-22T03:27:05.171767+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"188'1435","reported_seq":1546,"reported_epoch":188,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:06.204877+0000","last_change":"2024-09-22T03:27:04.476260+0000","last_active":"2024-09-22T03:27:06.204877+0000","last_peered":"2024-09-22T03:27:06.204877+0000","last_clean":"2024-09-22T03:25:30.049680+0000","last_became_active":"2024-09-22T03:27:04.473583+0000","last_became_peered":"2024-09-22T03:27:04.473583+0000","last_unstale":"2024-09-22T03:27:06.204877+0000","last_undegraded":"2024-09-22T03:27:04.294867+0000","last_fullsized":"2024-09-22T03:27:04.275186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":858}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":313,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-22T03:27:05.168434+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:05.168434+0000","last_peered":"2024-09-22T03:27:05.168434+0000","last_clean":"2024-09-22T03:27:05.168434+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:05.168434+0000","last_undegraded":"2024-09-22T03:27:05.168434+0000","last_fullsized":"2024-09-22T03:27:05.168434+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":866385944,"num_objects":13221,"num_object_clones":0,"num_object_copies":39663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":22827,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175841280,"data_stored":2813263944,"data_compressed":133588074,"data_compressed_allocated":175828992,"data_compressed_original":2813263872,"omap_allocated":0,"internal_metadata":0},"log_size":22813,"ondisk_log_size":22813,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":118528,"kb_used_data":37744,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":94253312,"statfs":{"total":96636764160,"available":96515391488,"internally_reserved":0,"allocated":38649856,"data_stored":593879557,"data_compressed":28169824,"data_compressed_allocated":37076992,"data_compressed_original":593231872,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949765,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":80308,"kb_used_data":46692,"kb_used_omap":0,"kb_used_meta":33600,"kb_avail":94291532,"statfs":{"total":96636764160,"available":96554528768,"internally_reserved":0,"allocated":47812608,"data_stored":740283909,"data_compressed":35123687,"data_compressed_allocated":46231552,"data_compressed_original":739622912,"omap_allocated":0,"internal_metadata":34406400},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102676,"kb_used_data":45376,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94269164,"statfs":{"total":96636764160,"available":96531623936,"internally_reserved":0,"allocated":46465024,"data_stored":718657053,"data_compressed":34097035,"data_compressed_allocated":44879872,"data_compressed_original":717996032,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95324,"kb_used_data":45452,"kb_used_omap":0,"kb_used_meta":49856,"kb_avail":94276516,"statfs":{"total":96636764160,"available":96539152384,"internally_reserved":0,"allocated":46542848,"data_stored":719967754,"data_compressed":34159275,"data_compressed_allocated":44961792,"data_compressed_original":719306752,"omap_allocated":0,"internal_metadata":51052544},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44621824,"data_stored":713883672,"data_compressed":33899015,"data_compressed_allocated":44617728,"data_compressed_original":713883648,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43954176,"data_stored":703201304,"data_compressed":33391759,"data_compressed_allocated":43950080,"data_compressed_original":703201280,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50831360,"data_stored":813301760,"data_compressed":38619172,"data_compressed_allocated":50831360,"data_compressed_original":813301760,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36433920,"data_stored":582877208,"data_compressed":27678128,"data_compressed_allocated":36429824,"data_compressed_original":582877184,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:12.637 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:12.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:12.747+0000 7fa07b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:12.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:12.752+0000 7fa07b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.773 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:14.842 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:14.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:14.898+0000 7f66b6597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:14.907+0000 7f66b6597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:14.969+0000 7f7c2c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:14.974+0000 7f7c2c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.125 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:15.126 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:15.141 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":674,"stamp":"2024-09-22T03:27:14.134601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":802226763,"num_objects":12237,"num_object_clones":0,"num_object_copies":36711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":23954,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1616,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23905,"ondisk_log_size":23905,"up":75,"acting":68,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":384764,"kb_used_data":159036,"kb_used_omap":0,"kb_used_meta":225664,"kb_avail":377102596,"statfs":{"total":386547056640,"available":386153058304,"internally_reserved":0,"allocated":162852864,"data_stored":2504231119,"data_compressed":118793872,"data_compressed_allocated":156360704,"data_compressed_original":2501525504,"omap_allocated":0,"internal_metadata":231079936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-64749568,"num_objects":-988,"num_object_clones":0,"num_object_copies":-2964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":988,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":987,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002955"},"pg_stats":[{"pgid":"6.f","version":"194'1491","reported_seq":2034,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.274865+0000","last_change":"2024-09-22T03:27:12.177019+0000","last_active":"2024-09-22T03:27:12.274865+0000","last_peered":"2024-09-22T03:27:12.274865+0000","last_clean":"2024-09-22T03:27:12.274865+0000","last_became_active":"2024-09-22T03:27:12.165305+0000","last_became_peered":"2024-09-22T03:27:12.165305+0000","last_unstale":"2024-09-22T03:27:12.274865+0000","last_undegraded":"2024-09-22T03:27:12.274865+0000","last_fullsized":"2024-09-22T03:27:12.274865+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1491,"log_dups_size":0,"ondisk_log_size":1491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":768,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1492,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"193'1492","reported_seq":2054,"reported_epoch":193,"state":"active+undersized+remapped","last_fresh":"2024-09-22T03:27:11.927143+0000","last_change":"2024-09-22T03:27:11.928140+0000","last_active":"2024-09-22T03:27:11.927143+0000","last_peered":"2024-09-22T03:27:11.927143+0000","last_clean":"2024-09-22T03:25:29.946334+0000","last_became_active":"2024-09-22T03:27:04.436044+0000","last_became_peered":"2024-09-22T03:27:04.436044+0000","last_unstale":"2024-09-22T03:27:11.927143+0000","last_undegraded":"2024-09-22T03:27:11.927143+0000","last_fullsized":"2024-09-22T03:27:04.181958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1492,"log_dups_size":0,"ondisk_log_size":1492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50987008,"num_objects":778,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1492,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,0],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"0,1","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"193'1447","reported_seq":1581,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.348426+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:11.348426+0000","last_peered":"2024-09-22T03:27:11.348426+0000","last_clean":"2024-09-22T03:27:11.348426+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:11.348426+0000","last_undegraded":"2024-09-22T03:27:11.348426+0000","last_fullsized":"2024-09-22T03:27:11.348426+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47906840,"num_objects":732,"num_object_clones":0,"num_object_copies":2196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1448,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"194'1539","reported_seq":2094,"reported_epoch":195,"state":"peering","last_fresh":"2024-09-22T03:27:12.842408+0000","last_change":"2024-09-22T03:27:13.146041+0000","last_active":"2024-09-22T03:27:13.146041+0000","last_peered":"2024-09-22T03:27:12.842408+0000","last_clean":"2024-09-22T03:26:47.209609+0000","last_became_active":"2024-09-22T03:27:04.335895+0000","last_became_peered":"2024-09-22T03:27:04.335895+0000","last_unstale":"2024-09-22T03:27:12.842408+0000","last_undegraded":"2024-09-22T03:27:12.842408+0000","last_fullsized":"2024-09-22T03:27:04.181884+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1539,"log_dups_size":0,"ondisk_log_size":1539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51838976,"num_objects":791,"num_object_clones":0,"num_object_copies":2373,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1539,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"193'1515","reported_seq":1859,"reported_epoch":193,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:12.115665+0000","last_change":"2024-09-22T03:27:10.637702+0000","last_active":"2024-09-22T03:27:12.115665+0000","last_peered":"2024-09-22T03:27:12.115665+0000","last_clean":"2024-09-22T03:26:47.187237+0000","last_became_active":"2024-09-22T03:27:04.374674+0000","last_became_peered":"2024-09-22T03:27:04.374674+0000","last_unstale":"2024-09-22T03:27:12.115665+0000","last_undegraded":"2024-09-22T03:27:12.115665+0000","last_fullsized":"2024-09-22T03:27:04.158651+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1515,"log_dups_size":0,"ondisk_log_size":1515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50528256,"num_objects":771,"num_object_clones":0,"num_object_copies":2313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1515,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":771}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"193'1460","reported_seq":1574,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.311309+0000","last_change":"2024-09-22T03:27:04.465312+0000","last_active":"2024-09-22T03:27:11.311309+0000","last_peered":"2024-09-22T03:27:11.311309+0000","last_clean":"2024-09-22T03:25:30.104671+0000","last_became_active":"2024-09-22T03:27:04.461526+0000","last_became_peered":"2024-09-22T03:27:04.461526+0000","last_unstale":"2024-09-22T03:27:11.311309+0000","last_undegraded":"2024-09-22T03:27:04.273968+0000","last_fullsized":"2024-09-22T03:27:04.255541+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1460,"log_dups_size":0,"ondisk_log_size":1460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49217536,"num_objects":751,"num_object_clones":0,"num_object_copies":2253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1461,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":751}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"194'1471","reported_seq":1572,"reported_epoch":195,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:13.140015+0000","last_change":"2024-09-22T03:27:04.507740+0000","last_active":"2024-09-22T03:27:13.140015+0000","last_peered":"2024-09-22T03:27:13.140015+0000","last_clean":"2024-09-22T03:26:47.207978+0000","last_became_active":"2024-09-22T03:27:04.298212+0000","last_became_peered":"2024-09-22T03:27:04.298212+0000","last_unstale":"2024-09-22T03:27:13.140015+0000","last_undegraded":"2024-09-22T03:27:04.505944+0000","last_fullsized":"2024-09-22T03:27:04.158046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1471,"log_dups_size":0,"ondisk_log_size":1471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47382528,"num_objects":723,"num_object_clones":0,"num_object_copies":2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1471,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":723}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"193'1407","reported_seq":1929,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.444343+0000","last_change":"2024-09-22T03:27:11.210627+0000","last_active":"2024-09-22T03:27:11.444343+0000","last_peered":"2024-09-22T03:27:11.444343+0000","last_clean":"2024-09-22T03:27:11.444343+0000","last_became_active":"2024-09-22T03:27:11.173122+0000","last_became_peered":"2024-09-22T03:27:11.173122+0000","last_unstale":"2024-09-22T03:27:11.444343+0000","last_undegraded":"2024-09-22T03:27:11.444343+0000","last_fullsized":"2024-09-22T03:27:11.444343+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1407,"log_dups_size":0,"ondisk_log_size":1407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1408,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"193'1411","reported_seq":1514,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.250904+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:11.250904+0000","last_peered":"2024-09-22T03:27:11.250904+0000","last_clean":"2024-09-22T03:27:11.250904+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:11.250904+0000","last_undegraded":"2024-09-22T03:27:11.250904+0000","last_fullsized":"2024-09-22T03:27:11.250904+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49414144,"num_objects":754,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":358,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.144918+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:13.144918+0000","last_peered":"2024-09-22T03:27:13.144918+0000","last_clean":"2024-09-22T03:27:13.144918+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:13.144918+0000","last_undegraded":"2024-09-22T03:27:13.144918+0000","last_fullsized":"2024-09-22T03:27:13.144918+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"194'1520","reported_seq":1670,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.145765+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:13.145765+0000","last_peered":"2024-09-22T03:27:13.145765+0000","last_clean":"2024-09-22T03:27:13.145765+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:13.145765+0000","last_undegraded":"2024-09-22T03:27:13.145765+0000","last_fullsized":"2024-09-22T03:27:13.145765+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1520,"log_dups_size":0,"ondisk_log_size":1520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52690944,"num_objects":804,"num_object_clones":0,"num_object_copies":2412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1520,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":458,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.147942+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:11.147942+0000","last_peered":"2024-09-22T03:27:11.147942+0000","last_clean":"2024-09-22T03:27:11.147942+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:11.147942+0000","last_undegraded":"2024-09-22T03:27:11.147942+0000","last_fullsized":"2024-09-22T03:27:11.147942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"193'1479","reported_seq":1611,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.500082+0000","last_change":"2024-09-22T03:27:04.479132+0000","last_active":"2024-09-22T03:27:11.500082+0000","last_peered":"2024-09-22T03:27:11.500082+0000","last_clean":"2024-09-22T03:25:30.080890+0000","last_became_active":"2024-09-22T03:27:04.474157+0000","last_became_peered":"2024-09-22T03:27:04.474157+0000","last_unstale":"2024-09-22T03:27:11.500082+0000","last_undegraded":"2024-09-22T03:27:04.292930+0000","last_fullsized":"2024-09-22T03:27:04.261632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1479,"log_dups_size":0,"ondisk_log_size":1479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51773440,"num_objects":790,"num_object_clones":0,"num_object_copies":2370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":790}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":477,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.149092+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:12.149092+0000","last_peered":"2024-09-22T03:27:12.149092+0000","last_clean":"2024-09-22T03:27:12.149092+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:12.149092+0000","last_undegraded":"2024-09-22T03:27:12.149092+0000","last_fullsized":"2024-09-22T03:27:12.149092+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"194'1535","reported_seq":1691,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.139388+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:13.139388+0000","last_peered":"2024-09-22T03:27:13.139388+0000","last_clean":"2024-09-22T03:27:13.139388+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:13.139388+0000","last_undegraded":"2024-09-22T03:27:13.139388+0000","last_fullsized":"2024-09-22T03:27:13.139388+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1535,"log_dups_size":0,"ondisk_log_size":1535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49217536,"num_objects":751,"num_object_clones":0,"num_object_copies":2253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1535,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.145314+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:13.145314+0000","last_peered":"2024-09-22T03:27:13.145314+0000","last_clean":"2024-09-22T03:27:13.145314+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:13.145314+0000","last_undegraded":"2024-09-22T03:27:13.145314+0000","last_fullsized":"2024-09-22T03:27:13.145314+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":576,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.155312+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:11.155312+0000","last_peered":"2024-09-22T03:27:11.155312+0000","last_clean":"2024-09-22T03:27:11.155312+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:11.155312+0000","last_undegraded":"2024-09-22T03:27:11.155312+0000","last_fullsized":"2024-09-22T03:27:11.155312+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"193'1493","reported_seq":1588,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.366996+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:11.366996+0000","last_peered":"2024-09-22T03:27:11.366996+0000","last_clean":"2024-09-22T03:27:11.366996+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:11.366996+0000","last_undegraded":"2024-09-22T03:27:11.366996+0000","last_fullsized":"2024-09-22T03:27:11.366996+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1493,"log_dups_size":0,"ondisk_log_size":1493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50987008,"num_objects":778,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1494,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":251,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.151671+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:11.151671+0000","last_peered":"2024-09-22T03:27:11.151671+0000","last_clean":"2024-09-22T03:27:11.151671+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:11.151671+0000","last_undegraded":"2024-09-22T03:27:11.151671+0000","last_fullsized":"2024-09-22T03:27:11.151671+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"193'1486","reported_seq":1613,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.477518+0000","last_change":"2024-09-22T03:27:04.479082+0000","last_active":"2024-09-22T03:27:11.477518+0000","last_peered":"2024-09-22T03:27:11.477518+0000","last_clean":"2024-09-22T03:25:30.070791+0000","last_became_active":"2024-09-22T03:27:04.472230+0000","last_became_peered":"2024-09-22T03:27:04.472230+0000","last_unstale":"2024-09-22T03:27:11.477518+0000","last_undegraded":"2024-09-22T03:27:04.203862+0000","last_fullsized":"2024-09-22T03:27:04.183417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48824320,"num_objects":745,"num_object_clones":0,"num_object_copies":2235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":745}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":374,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.143989+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:12.143989+0000","last_peered":"2024-09-22T03:27:12.143989+0000","last_clean":"2024-09-22T03:27:12.143989+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:12.143989+0000","last_undegraded":"2024-09-22T03:27:12.143989+0000","last_fullsized":"2024-09-22T03:27:12.143989+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"194'1547","reported_seq":2150,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.509647+0000","last_change":"2024-09-22T03:27:10.180074+0000","last_active":"2024-09-22T03:27:12.509647+0000","last_peered":"2024-09-22T03:27:12.509647+0000","last_clean":"2024-09-22T03:27:12.509647+0000","last_became_active":"2024-09-22T03:27:10.168804+0000","last_became_peered":"2024-09-22T03:27:10.168804+0000","last_unstale":"2024-09-22T03:27:12.509647+0000","last_undegraded":"2024-09-22T03:27:12.509647+0000","last_fullsized":"2024-09-22T03:27:12.509647+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1547,"log_dups_size":0,"ondisk_log_size":1547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51118080,"num_objects":780,"num_object_clones":0,"num_object_copies":2340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1548,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.157859+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:11.157859+0000","last_peered":"2024-09-22T03:27:11.157859+0000","last_clean":"2024-09-22T03:27:11.157859+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:11.157859+0000","last_undegraded":"2024-09-22T03:27:11.157859+0000","last_fullsized":"2024-09-22T03:27:11.157859+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"193'1512","reported_seq":1633,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.355092+0000","last_change":"2024-09-22T03:27:04.476260+0000","last_active":"2024-09-22T03:27:11.355092+0000","last_peered":"2024-09-22T03:27:11.355092+0000","last_clean":"2024-09-22T03:25:30.049680+0000","last_became_active":"2024-09-22T03:27:04.473583+0000","last_became_peered":"2024-09-22T03:27:04.473583+0000","last_unstale":"2024-09-22T03:27:11.355092+0000","last_undegraded":"2024-09-22T03:27:04.294867+0000","last_fullsized":"2024-09-22T03:27:04.275186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1512,"log_dups_size":0,"ondisk_log_size":1512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51183616,"num_objects":781,"num_object_clones":0,"num_object_copies":2343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1513,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":781}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":328,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.157050+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:11.157050+0000","last_peered":"2024-09-22T03:27:11.157050+0000","last_clean":"2024-09-22T03:27:11.157050+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:11.157050+0000","last_undegraded":"2024-09-22T03:27:11.157050+0000","last_fullsized":"2024-09-22T03:27:11.157050+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":801636376,"num_objects":12233,"num_object_clones":0,"num_object_copies":36699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":23815,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1616,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154791936,"data_stored":2476474440,"data_compressed":117596121,"data_compressed_allocated":154779648,"data_compressed_original":2476474368,"omap_allocated":0,"internal_metadata":0},"log_size":23805,"ondisk_log_size":23805,"up":48,"acting":41,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115924,"kb_used_data":35140,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":94255916,"statfs":{"total":96636764160,"available":96518057984,"internally_reserved":0,"allocated":35983360,"data_stored":550578047,"data_compressed":26112792,"data_compressed_allocated":34369536,"data_compressed_original":549912576,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86592,"kb_used_data":40496,"kb_used_omap":0,"kb_used_meta":46080,"kb_avail":94285248,"statfs":{"total":96636764160,"available":96548093952,"internally_reserved":0,"allocated":41467904,"data_stored":638003591,"data_compressed":30265994,"data_compressed_allocated":39837696,"data_compressed_original":637321216,"omap_allocated":0,"internal_metadata":47185920},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103160,"kb_used_data":41704,"kb_used_omap":0,"kb_used_meta":61440,"kb_avail":94268680,"statfs":{"total":96636764160,"available":96531128320,"internally_reserved":0,"allocated":42704896,"data_stored":657857518,"data_compressed":31209099,"data_compressed_allocated":41078784,"data_compressed_original":657178624,"omap_allocated":0,"internal_metadata":62914560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79088,"kb_used_data":41696,"kb_used_omap":0,"kb_used_meta":37376,"kb_avail":94292752,"statfs":{"total":96636764160,"available":96555778048,"internally_reserved":0,"allocated":42696704,"data_stored":657791963,"data_compressed":31205987,"data_compressed_allocated":41074688,"data_compressed_original":657113088,"omap_allocated":0,"internal_metadata":38273024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40890368,"data_stored":654180376,"data_compressed":31063983,"data_compressed_allocated":40886272,"data_compressed_original":654180352,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40849408,"data_stored":653525016,"data_compressed":31032863,"data_compressed_allocated":40845312,"data_compressed_original":653524992,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":619511808,"data_compressed":29417603,"data_compressed_allocated":38719488,"data_compressed_original":619511808,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34332672,"data_stored":549257240,"data_compressed":26081672,"data_compressed_allocated":34328576,"data_compressed_original":549257216,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:15.143 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:27:15.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:15.317+0000 7fbcc0197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:15.329+0000 7fbcc0197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.725 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:15.725 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1129/36711 objects degraded (3.075%), 5 pgs degraded","count":5},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":980,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":197,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975622,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+remapped","count":1},{"state_name":"active+undersized+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":25,"num_pools":3,"num_objects":12237,"data_bytes":802226763,"bytes_used":393998336,"bytes_avail":386153058304,"bytes_total":386547056640,"inactive_pgs_ratio":0.039999999105930328,"degraded_objects":1129,"degraded_total":36711,"degraded_ratio":0.030753725041540683,"recovering_objects_per_sec":246,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":987,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":246},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T03:27:01.429211+0000","services":{}},"progress_events":{}} 2024-09-22T03:27:15.739 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:27:15.739 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:15.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:15.852+0000 7f22fca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:15.860+0000 7f22fca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.077 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:16.077 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:16.091 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":676,"stamp":"2024-09-22T03:27:15.432140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":802226763,"num_objects":12237,"num_object_clones":0,"num_object_copies":36711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":23954,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1616,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23905,"ondisk_log_size":23905,"up":75,"acting":68,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":384764,"kb_used_data":159036,"kb_used_omap":0,"kb_used_meta":225664,"kb_avail":377102596,"statfs":{"total":386547056640,"available":386153058304,"internally_reserved":0,"allocated":162852864,"data_stored":2504231119,"data_compressed":118793872,"data_compressed_allocated":156360704,"data_compressed_original":2501525504,"omap_allocated":0,"internal_metadata":231079936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-64749568,"num_objects":-988,"num_object_clones":0,"num_object_copies":-2964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":988,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":987,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000820"},"pg_stats":[{"pgid":"6.f","version":"194'1491","reported_seq":2034,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.274865+0000","last_change":"2024-09-22T03:27:12.177019+0000","last_active":"2024-09-22T03:27:12.274865+0000","last_peered":"2024-09-22T03:27:12.274865+0000","last_clean":"2024-09-22T03:27:12.274865+0000","last_became_active":"2024-09-22T03:27:12.165305+0000","last_became_peered":"2024-09-22T03:27:12.165305+0000","last_unstale":"2024-09-22T03:27:12.274865+0000","last_undegraded":"2024-09-22T03:27:12.274865+0000","last_fullsized":"2024-09-22T03:27:12.274865+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1491,"log_dups_size":0,"ondisk_log_size":1491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":768,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1492,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"193'1492","reported_seq":2054,"reported_epoch":193,"state":"active+undersized+remapped","last_fresh":"2024-09-22T03:27:11.927143+0000","last_change":"2024-09-22T03:27:11.928140+0000","last_active":"2024-09-22T03:27:11.927143+0000","last_peered":"2024-09-22T03:27:11.927143+0000","last_clean":"2024-09-22T03:25:29.946334+0000","last_became_active":"2024-09-22T03:27:04.436044+0000","last_became_peered":"2024-09-22T03:27:04.436044+0000","last_unstale":"2024-09-22T03:27:11.927143+0000","last_undegraded":"2024-09-22T03:27:11.927143+0000","last_fullsized":"2024-09-22T03:27:04.181958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1492,"log_dups_size":0,"ondisk_log_size":1492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50987008,"num_objects":778,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1492,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,0],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"0,1","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"193'1447","reported_seq":1581,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.348426+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:11.348426+0000","last_peered":"2024-09-22T03:27:11.348426+0000","last_clean":"2024-09-22T03:27:11.348426+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:11.348426+0000","last_undegraded":"2024-09-22T03:27:11.348426+0000","last_fullsized":"2024-09-22T03:27:11.348426+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47906840,"num_objects":732,"num_object_clones":0,"num_object_copies":2196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1448,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"194'1539","reported_seq":2094,"reported_epoch":195,"state":"peering","last_fresh":"2024-09-22T03:27:12.842408+0000","last_change":"2024-09-22T03:27:13.146041+0000","last_active":"2024-09-22T03:27:13.146041+0000","last_peered":"2024-09-22T03:27:12.842408+0000","last_clean":"2024-09-22T03:26:47.209609+0000","last_became_active":"2024-09-22T03:27:04.335895+0000","last_became_peered":"2024-09-22T03:27:04.335895+0000","last_unstale":"2024-09-22T03:27:12.842408+0000","last_undegraded":"2024-09-22T03:27:12.842408+0000","last_fullsized":"2024-09-22T03:27:04.181884+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1539,"log_dups_size":0,"ondisk_log_size":1539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51838976,"num_objects":791,"num_object_clones":0,"num_object_copies":2373,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1539,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"193'1515","reported_seq":1859,"reported_epoch":193,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:12.115665+0000","last_change":"2024-09-22T03:27:10.637702+0000","last_active":"2024-09-22T03:27:12.115665+0000","last_peered":"2024-09-22T03:27:12.115665+0000","last_clean":"2024-09-22T03:26:47.187237+0000","last_became_active":"2024-09-22T03:27:04.374674+0000","last_became_peered":"2024-09-22T03:27:04.374674+0000","last_unstale":"2024-09-22T03:27:12.115665+0000","last_undegraded":"2024-09-22T03:27:12.115665+0000","last_fullsized":"2024-09-22T03:27:04.158651+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1515,"log_dups_size":0,"ondisk_log_size":1515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50528256,"num_objects":771,"num_object_clones":0,"num_object_copies":2313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1515,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":771}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"193'1460","reported_seq":1574,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.311309+0000","last_change":"2024-09-22T03:27:04.465312+0000","last_active":"2024-09-22T03:27:11.311309+0000","last_peered":"2024-09-22T03:27:11.311309+0000","last_clean":"2024-09-22T03:25:30.104671+0000","last_became_active":"2024-09-22T03:27:04.461526+0000","last_became_peered":"2024-09-22T03:27:04.461526+0000","last_unstale":"2024-09-22T03:27:11.311309+0000","last_undegraded":"2024-09-22T03:27:04.273968+0000","last_fullsized":"2024-09-22T03:27:04.255541+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1460,"log_dups_size":0,"ondisk_log_size":1460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49217536,"num_objects":751,"num_object_clones":0,"num_object_copies":2253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1461,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":751}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"194'1471","reported_seq":1572,"reported_epoch":195,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:13.140015+0000","last_change":"2024-09-22T03:27:04.507740+0000","last_active":"2024-09-22T03:27:13.140015+0000","last_peered":"2024-09-22T03:27:13.140015+0000","last_clean":"2024-09-22T03:26:47.207978+0000","last_became_active":"2024-09-22T03:27:04.298212+0000","last_became_peered":"2024-09-22T03:27:04.298212+0000","last_unstale":"2024-09-22T03:27:13.140015+0000","last_undegraded":"2024-09-22T03:27:04.505944+0000","last_fullsized":"2024-09-22T03:27:04.158046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1471,"log_dups_size":0,"ondisk_log_size":1471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47382528,"num_objects":723,"num_object_clones":0,"num_object_copies":2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1471,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":723}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"193'1407","reported_seq":1929,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.444343+0000","last_change":"2024-09-22T03:27:11.210627+0000","last_active":"2024-09-22T03:27:11.444343+0000","last_peered":"2024-09-22T03:27:11.444343+0000","last_clean":"2024-09-22T03:27:11.444343+0000","last_became_active":"2024-09-22T03:27:11.173122+0000","last_became_peered":"2024-09-22T03:27:11.173122+0000","last_unstale":"2024-09-22T03:27:11.444343+0000","last_undegraded":"2024-09-22T03:27:11.444343+0000","last_fullsized":"2024-09-22T03:27:11.444343+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1407,"log_dups_size":0,"ondisk_log_size":1407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1408,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"193'1411","reported_seq":1514,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.250904+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:11.250904+0000","last_peered":"2024-09-22T03:27:11.250904+0000","last_clean":"2024-09-22T03:27:11.250904+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:11.250904+0000","last_undegraded":"2024-09-22T03:27:11.250904+0000","last_fullsized":"2024-09-22T03:27:11.250904+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49414144,"num_objects":754,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1412,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":358,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.144918+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:13.144918+0000","last_peered":"2024-09-22T03:27:13.144918+0000","last_clean":"2024-09-22T03:27:13.144918+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:13.144918+0000","last_undegraded":"2024-09-22T03:27:13.144918+0000","last_fullsized":"2024-09-22T03:27:13.144918+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"194'1520","reported_seq":1670,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.145765+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:13.145765+0000","last_peered":"2024-09-22T03:27:13.145765+0000","last_clean":"2024-09-22T03:27:13.145765+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:13.145765+0000","last_undegraded":"2024-09-22T03:27:13.145765+0000","last_fullsized":"2024-09-22T03:27:13.145765+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1520,"log_dups_size":0,"ondisk_log_size":1520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52690944,"num_objects":804,"num_object_clones":0,"num_object_copies":2412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1520,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":458,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.147942+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:11.147942+0000","last_peered":"2024-09-22T03:27:11.147942+0000","last_clean":"2024-09-22T03:27:11.147942+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:11.147942+0000","last_undegraded":"2024-09-22T03:27:11.147942+0000","last_fullsized":"2024-09-22T03:27:11.147942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"193'1479","reported_seq":1611,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.500082+0000","last_change":"2024-09-22T03:27:04.479132+0000","last_active":"2024-09-22T03:27:11.500082+0000","last_peered":"2024-09-22T03:27:11.500082+0000","last_clean":"2024-09-22T03:25:30.080890+0000","last_became_active":"2024-09-22T03:27:04.474157+0000","last_became_peered":"2024-09-22T03:27:04.474157+0000","last_unstale":"2024-09-22T03:27:11.500082+0000","last_undegraded":"2024-09-22T03:27:04.292930+0000","last_fullsized":"2024-09-22T03:27:04.261632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1479,"log_dups_size":0,"ondisk_log_size":1479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51773440,"num_objects":790,"num_object_clones":0,"num_object_copies":2370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":790}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":477,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.149092+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:12.149092+0000","last_peered":"2024-09-22T03:27:12.149092+0000","last_clean":"2024-09-22T03:27:12.149092+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:12.149092+0000","last_undegraded":"2024-09-22T03:27:12.149092+0000","last_fullsized":"2024-09-22T03:27:12.149092+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"194'1535","reported_seq":1691,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.139388+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:13.139388+0000","last_peered":"2024-09-22T03:27:13.139388+0000","last_clean":"2024-09-22T03:27:13.139388+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:13.139388+0000","last_undegraded":"2024-09-22T03:27:13.139388+0000","last_fullsized":"2024-09-22T03:27:13.139388+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1535,"log_dups_size":0,"ondisk_log_size":1535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49217536,"num_objects":751,"num_object_clones":0,"num_object_copies":2253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1535,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T03:27:13.145314+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:13.145314+0000","last_peered":"2024-09-22T03:27:13.145314+0000","last_clean":"2024-09-22T03:27:13.145314+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:13.145314+0000","last_undegraded":"2024-09-22T03:27:13.145314+0000","last_fullsized":"2024-09-22T03:27:13.145314+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":576,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.155312+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:11.155312+0000","last_peered":"2024-09-22T03:27:11.155312+0000","last_clean":"2024-09-22T03:27:11.155312+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:11.155312+0000","last_undegraded":"2024-09-22T03:27:11.155312+0000","last_fullsized":"2024-09-22T03:27:11.155312+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"193'1493","reported_seq":1588,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.366996+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:11.366996+0000","last_peered":"2024-09-22T03:27:11.366996+0000","last_clean":"2024-09-22T03:27:11.366996+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:11.366996+0000","last_undegraded":"2024-09-22T03:27:11.366996+0000","last_fullsized":"2024-09-22T03:27:11.366996+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1493,"log_dups_size":0,"ondisk_log_size":1493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50987008,"num_objects":778,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1494,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":251,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.151671+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:11.151671+0000","last_peered":"2024-09-22T03:27:11.151671+0000","last_clean":"2024-09-22T03:27:11.151671+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:11.151671+0000","last_undegraded":"2024-09-22T03:27:11.151671+0000","last_fullsized":"2024-09-22T03:27:11.151671+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"193'1486","reported_seq":1613,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.477518+0000","last_change":"2024-09-22T03:27:04.479082+0000","last_active":"2024-09-22T03:27:11.477518+0000","last_peered":"2024-09-22T03:27:11.477518+0000","last_clean":"2024-09-22T03:25:30.070791+0000","last_became_active":"2024-09-22T03:27:04.472230+0000","last_became_peered":"2024-09-22T03:27:04.472230+0000","last_unstale":"2024-09-22T03:27:11.477518+0000","last_undegraded":"2024-09-22T03:27:04.203862+0000","last_fullsized":"2024-09-22T03:27:04.183417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48824320,"num_objects":745,"num_object_clones":0,"num_object_copies":2235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":745}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":374,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.143989+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:12.143989+0000","last_peered":"2024-09-22T03:27:12.143989+0000","last_clean":"2024-09-22T03:27:12.143989+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:12.143989+0000","last_undegraded":"2024-09-22T03:27:12.143989+0000","last_fullsized":"2024-09-22T03:27:12.143989+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"194'1547","reported_seq":2150,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T03:27:12.509647+0000","last_change":"2024-09-22T03:27:10.180074+0000","last_active":"2024-09-22T03:27:12.509647+0000","last_peered":"2024-09-22T03:27:12.509647+0000","last_clean":"2024-09-22T03:27:12.509647+0000","last_became_active":"2024-09-22T03:27:10.168804+0000","last_became_peered":"2024-09-22T03:27:10.168804+0000","last_unstale":"2024-09-22T03:27:12.509647+0000","last_undegraded":"2024-09-22T03:27:12.509647+0000","last_fullsized":"2024-09-22T03:27:12.509647+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1547,"log_dups_size":0,"ondisk_log_size":1547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51118080,"num_objects":780,"num_object_clones":0,"num_object_copies":2340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1548,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.157859+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:11.157859+0000","last_peered":"2024-09-22T03:27:11.157859+0000","last_clean":"2024-09-22T03:27:11.157859+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:11.157859+0000","last_undegraded":"2024-09-22T03:27:11.157859+0000","last_fullsized":"2024-09-22T03:27:11.157859+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"193'1512","reported_seq":1633,"reported_epoch":193,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:27:11.355092+0000","last_change":"2024-09-22T03:27:04.476260+0000","last_active":"2024-09-22T03:27:11.355092+0000","last_peered":"2024-09-22T03:27:11.355092+0000","last_clean":"2024-09-22T03:25:30.049680+0000","last_became_active":"2024-09-22T03:27:04.473583+0000","last_became_peered":"2024-09-22T03:27:04.473583+0000","last_unstale":"2024-09-22T03:27:11.355092+0000","last_undegraded":"2024-09-22T03:27:04.294867+0000","last_fullsized":"2024-09-22T03:27:04.275186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1512,"log_dups_size":0,"ondisk_log_size":1512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51183616,"num_objects":781,"num_object_clones":0,"num_object_copies":2343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1513,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":781}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":328,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.157050+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:11.157050+0000","last_peered":"2024-09-22T03:27:11.157050+0000","last_clean":"2024-09-22T03:27:11.157050+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:11.157050+0000","last_undegraded":"2024-09-22T03:27:11.157050+0000","last_fullsized":"2024-09-22T03:27:11.157050+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":801636376,"num_objects":12233,"num_object_clones":0,"num_object_copies":36699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":23815,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1616,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154791936,"data_stored":2476474440,"data_compressed":117596121,"data_compressed_allocated":154779648,"data_compressed_original":2476474368,"omap_allocated":0,"internal_metadata":0},"log_size":23805,"ondisk_log_size":23805,"up":48,"acting":41,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115924,"kb_used_data":35140,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":94255916,"statfs":{"total":96636764160,"available":96518057984,"internally_reserved":0,"allocated":35983360,"data_stored":550578047,"data_compressed":26112792,"data_compressed_allocated":34369536,"data_compressed_original":549912576,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86592,"kb_used_data":40496,"kb_used_omap":0,"kb_used_meta":46080,"kb_avail":94285248,"statfs":{"total":96636764160,"available":96548093952,"internally_reserved":0,"allocated":41467904,"data_stored":638003591,"data_compressed":30265994,"data_compressed_allocated":39837696,"data_compressed_original":637321216,"omap_allocated":0,"internal_metadata":47185920},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103160,"kb_used_data":41704,"kb_used_omap":0,"kb_used_meta":61440,"kb_avail":94268680,"statfs":{"total":96636764160,"available":96531128320,"internally_reserved":0,"allocated":42704896,"data_stored":657857518,"data_compressed":31209099,"data_compressed_allocated":41078784,"data_compressed_original":657178624,"omap_allocated":0,"internal_metadata":62914560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79088,"kb_used_data":41696,"kb_used_omap":0,"kb_used_meta":37376,"kb_avail":94292752,"statfs":{"total":96636764160,"available":96555778048,"internally_reserved":0,"allocated":42696704,"data_stored":657791963,"data_compressed":31205987,"data_compressed_allocated":41074688,"data_compressed_original":657113088,"omap_allocated":0,"internal_metadata":38273024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40890368,"data_stored":654180376,"data_compressed":31063983,"data_compressed_allocated":40886272,"data_compressed_original":654180352,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40849408,"data_stored":653525016,"data_compressed":31032863,"data_compressed_allocated":40845312,"data_compressed_original":653524992,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":619511808,"data_compressed":29417603,"data_compressed_allocated":38719488,"data_compressed_original":619511808,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34332672,"data_stored":549257240,"data_compressed":26081672,"data_compressed_allocated":34328576,"data_compressed_original":549257216,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:17.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:17.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:17.233+0000 7fdf54f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:17.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:17.239+0000 7fdf54f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.093 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:19.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:19.209+0000 7f2931797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:19.217+0000 7f2931797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.472 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:19.473 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:19.474 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:19.491 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":682,"stamp":"2024-09-22T03:27:19.432808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":762053195,"num_objects":11624,"num_object_clones":0,"num_object_copies":34872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":24567,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2662,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24515,"ondisk_log_size":24515,"up":75,"acting":73,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":357072,"kb_used_data":147980,"kb_used_omap":0,"kb_used_meta":209024,"kb_avail":377130288,"statfs":{"total":386547056640,"available":386181414912,"internally_reserved":0,"allocated":151531520,"data_stored":2320536825,"data_compressed":110067993,"data_compressed_allocated":144875520,"data_compressed_original":2317762560,"omap_allocated":0,"internal_metadata":214040576},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-40173568,"num_objects":-613,"num_object_clones":0,"num_object_copies":-1839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":613,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000667"},"pg_stats":[{"pgid":"6.f","version":"199'1534","reported_seq":2092,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.301309+0000","last_change":"2024-09-22T03:27:12.177019+0000","last_active":"2024-09-22T03:27:17.301309+0000","last_peered":"2024-09-22T03:27:17.301309+0000","last_clean":"2024-09-22T03:27:17.301309+0000","last_became_active":"2024-09-22T03:27:12.165305+0000","last_became_peered":"2024-09-22T03:27:12.165305+0000","last_unstale":"2024-09-22T03:27:17.301309+0000","last_undegraded":"2024-09-22T03:27:17.301309+0000","last_fullsized":"2024-09-22T03:27:17.301309+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1534,"log_dups_size":0,"ondisk_log_size":1534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47513600,"num_objects":725,"num_object_clones":0,"num_object_copies":2175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1535,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"198'1543","reported_seq":2124,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.723823+0000","last_change":"2024-09-22T03:27:13.172481+0000","last_active":"2024-09-22T03:27:16.723823+0000","last_peered":"2024-09-22T03:27:16.723823+0000","last_clean":"2024-09-22T03:27:16.723823+0000","last_became_active":"2024-09-22T03:27:13.160685+0000","last_became_peered":"2024-09-22T03:27:13.160685+0000","last_unstale":"2024-09-22T03:27:16.723823+0000","last_undegraded":"2024-09-22T03:27:16.723823+0000","last_fullsized":"2024-09-22T03:27:16.723823+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1543,"log_dups_size":0,"ondisk_log_size":1543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47579136,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1544,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"198'1477","reported_seq":1626,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.883830+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:16.883830+0000","last_peered":"2024-09-22T03:27:16.883830+0000","last_clean":"2024-09-22T03:27:16.883830+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:16.883830+0000","last_undegraded":"2024-09-22T03:27:16.883830+0000","last_fullsized":"2024-09-22T03:27:16.883830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1477,"log_dups_size":0,"ondisk_log_size":1477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45940760,"num_objects":702,"num_object_clones":0,"num_object_copies":2106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1478,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"199'1591","reported_seq":2164,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.525439+0000","last_change":"2024-09-22T03:27:14.173803+0000","last_active":"2024-09-22T03:27:17.525439+0000","last_peered":"2024-09-22T03:27:17.525439+0000","last_clean":"2024-09-22T03:27:17.525439+0000","last_became_active":"2024-09-22T03:27:14.167068+0000","last_became_peered":"2024-09-22T03:27:14.167068+0000","last_unstale":"2024-09-22T03:27:17.525439+0000","last_undegraded":"2024-09-22T03:27:17.525439+0000","last_fullsized":"2024-09-22T03:27:17.525439+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1591,"log_dups_size":0,"ondisk_log_size":1591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48365568,"num_objects":738,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1592,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"198'1559","reported_seq":2145,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.851952+0000","last_change":"2024-09-22T03:27:15.169223+0000","last_active":"2024-09-22T03:27:16.851952+0000","last_peered":"2024-09-22T03:27:16.851952+0000","last_clean":"2024-09-22T03:27:16.851952+0000","last_became_active":"2024-09-22T03:27:15.163663+0000","last_became_peered":"2024-09-22T03:27:15.163663+0000","last_unstale":"2024-09-22T03:27:16.851952+0000","last_undegraded":"2024-09-22T03:27:16.851952+0000","last_fullsized":"2024-09-22T03:27:16.851952+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1559,"log_dups_size":0,"ondisk_log_size":1559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47579136,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"199'1499","reported_seq":2071,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.313349+0000","last_change":"2024-09-22T03:27:16.205368+0000","last_active":"2024-09-22T03:27:17.313349+0000","last_peered":"2024-09-22T03:27:17.313349+0000","last_clean":"2024-09-22T03:27:17.313349+0000","last_became_active":"2024-09-22T03:27:16.194196+0000","last_became_peered":"2024-09-22T03:27:16.194196+0000","last_unstale":"2024-09-22T03:27:17.313349+0000","last_undegraded":"2024-09-22T03:27:17.313349+0000","last_fullsized":"2024-09-22T03:27:17.313349+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1499,"log_dups_size":0,"ondisk_log_size":1499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46661632,"num_objects":712,"num_object_clones":0,"num_object_copies":2136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1500,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"199'1511","reported_seq":1967,"reported_epoch":199,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:18.154332+0000","last_change":"2024-09-22T03:27:16.053905+0000","last_active":"2024-09-22T03:27:18.154332+0000","last_peered":"2024-09-22T03:27:18.154332+0000","last_clean":"2024-09-22T03:26:47.207978+0000","last_became_active":"2024-09-22T03:27:04.298212+0000","last_became_peered":"2024-09-22T03:27:04.298212+0000","last_unstale":"2024-09-22T03:27:18.154332+0000","last_undegraded":"2024-09-22T03:27:18.154332+0000","last_fullsized":"2024-09-22T03:27:04.158046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1511,"log_dups_size":0,"ondisk_log_size":1511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44761088,"num_objects":683,"num_object_clones":0,"num_object_copies":2049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1511,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":683}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"198'1448","reported_seq":1985,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.498382+0000","last_change":"2024-09-22T03:27:11.210627+0000","last_active":"2024-09-22T03:27:16.498382+0000","last_peered":"2024-09-22T03:27:16.498382+0000","last_clean":"2024-09-22T03:27:16.498382+0000","last_became_active":"2024-09-22T03:27:11.173122+0000","last_became_peered":"2024-09-22T03:27:11.173122+0000","last_unstale":"2024-09-22T03:27:16.498382+0000","last_undegraded":"2024-09-22T03:27:16.498382+0000","last_fullsized":"2024-09-22T03:27:16.498382+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1448,"log_dups_size":0,"ondisk_log_size":1448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45547520,"num_objects":695,"num_object_clones":0,"num_object_copies":2085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1449,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"198'1443","reported_seq":1561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.509580+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:16.509580+0000","last_peered":"2024-09-22T03:27:16.509580+0000","last_clean":"2024-09-22T03:27:16.509580+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:16.509580+0000","last_undegraded":"2024-09-22T03:27:16.509580+0000","last_fullsized":"2024-09-22T03:27:16.509580+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1443,"log_dups_size":0,"ondisk_log_size":1443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47316992,"num_objects":722,"num_object_clones":0,"num_object_copies":2166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1444,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":370,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.158390+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:17.158390+0000","last_peered":"2024-09-22T03:27:17.158390+0000","last_clean":"2024-09-22T03:27:17.158390+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:17.158390+0000","last_undegraded":"2024-09-22T03:27:17.158390+0000","last_fullsized":"2024-09-22T03:27:17.158390+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"199'1555","reported_seq":1718,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.485205+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:17.485205+0000","last_peered":"2024-09-22T03:27:17.485205+0000","last_clean":"2024-09-22T03:27:17.485205+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:17.485205+0000","last_undegraded":"2024-09-22T03:27:17.485205+0000","last_fullsized":"2024-09-22T03:27:17.485205+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":768,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":473,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.147481+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:16.147481+0000","last_peered":"2024-09-22T03:27:16.147481+0000","last_clean":"2024-09-22T03:27:16.147481+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:16.147481+0000","last_undegraded":"2024-09-22T03:27:16.147481+0000","last_fullsized":"2024-09-22T03:27:16.147481+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"198'1513","reported_seq":1858,"reported_epoch":198,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:16.489343+0000","last_change":"2024-09-22T03:27:14.794805+0000","last_active":"2024-09-22T03:27:16.489343+0000","last_peered":"2024-09-22T03:27:16.489343+0000","last_clean":"2024-09-22T03:25:30.080890+0000","last_became_active":"2024-09-22T03:27:04.474157+0000","last_became_peered":"2024-09-22T03:27:04.474157+0000","last_unstale":"2024-09-22T03:27:16.489343+0000","last_undegraded":"2024-09-22T03:27:16.489343+0000","last_fullsized":"2024-09-22T03:27:04.261632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1513,"log_dups_size":0,"ondisk_log_size":1513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49545216,"num_objects":756,"num_object_clones":0,"num_object_copies":2268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1514,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":756}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":492,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.160890+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:17.160890+0000","last_peered":"2024-09-22T03:27:17.160890+0000","last_clean":"2024-09-22T03:27:17.160890+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:17.160890+0000","last_undegraded":"2024-09-22T03:27:17.160890+0000","last_fullsized":"2024-09-22T03:27:17.160890+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"199'1572","reported_seq":1741,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.331088+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:17.331088+0000","last_peered":"2024-09-22T03:27:17.331088+0000","last_clean":"2024-09-22T03:27:17.331088+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:17.331088+0000","last_undegraded":"2024-09-22T03:27:17.331088+0000","last_fullsized":"2024-09-22T03:27:17.331088+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1573,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":456,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.157448+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:17.157448+0000","last_peered":"2024-09-22T03:27:17.157448+0000","last_clean":"2024-09-22T03:27:17.157448+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:17.157448+0000","last_undegraded":"2024-09-22T03:27:17.157448+0000","last_fullsized":"2024-09-22T03:27:17.157448+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":591,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.154875+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:16.154875+0000","last_peered":"2024-09-22T03:27:16.154875+0000","last_clean":"2024-09-22T03:27:16.154875+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:16.154875+0000","last_undegraded":"2024-09-22T03:27:16.154875+0000","last_fullsized":"2024-09-22T03:27:16.154875+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"198'1525","reported_seq":1635,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.325471+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:16.325471+0000","last_peered":"2024-09-22T03:27:16.325471+0000","last_clean":"2024-09-22T03:27:16.325471+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:16.325471+0000","last_undegraded":"2024-09-22T03:27:16.325471+0000","last_fullsized":"2024-09-22T03:27:16.325471+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1525,"log_dups_size":0,"ondisk_log_size":1525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1526,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":266,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.148186+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:16.148186+0000","last_peered":"2024-09-22T03:27:16.148186+0000","last_clean":"2024-09-22T03:27:16.148186+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:16.148186+0000","last_undegraded":"2024-09-22T03:27:16.148186+0000","last_fullsized":"2024-09-22T03:27:16.148186+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"197'1511","reported_seq":2128,"reported_epoch":198,"state":"peering","last_fresh":"2024-09-22T03:27:16.036241+0000","last_change":"2024-09-22T03:27:16.151387+0000","last_active":"2024-09-22T03:27:16.151387+0000","last_peered":"2024-09-22T03:27:16.036241+0000","last_clean":"2024-09-22T03:25:30.070791+0000","last_became_active":"2024-09-22T03:27:04.472230+0000","last_became_peered":"2024-09-22T03:27:04.472230+0000","last_unstale":"2024-09-22T03:27:16.036241+0000","last_undegraded":"2024-09-22T03:27:16.036241+0000","last_fullsized":"2024-09-22T03:27:04.183417+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1511,"log_dups_size":0,"ondisk_log_size":1511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47251456,"num_objects":721,"num_object_clones":0,"num_object_copies":2163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1511,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":389,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.154219+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:17.154219+0000","last_peered":"2024-09-22T03:27:17.154219+0000","last_clean":"2024-09-22T03:27:17.154219+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:17.154219+0000","last_undegraded":"2024-09-22T03:27:17.154219+0000","last_fullsized":"2024-09-22T03:27:17.154219+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"199'1581","reported_seq":2199,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.330882+0000","last_change":"2024-09-22T03:27:10.180074+0000","last_active":"2024-09-22T03:27:17.330882+0000","last_peered":"2024-09-22T03:27:17.330882+0000","last_clean":"2024-09-22T03:27:17.330882+0000","last_became_active":"2024-09-22T03:27:10.168804+0000","last_became_peered":"2024-09-22T03:27:10.168804+0000","last_unstale":"2024-09-22T03:27:17.330882+0000","last_undegraded":"2024-09-22T03:27:17.330882+0000","last_fullsized":"2024-09-22T03:27:17.330882+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1581,"log_dups_size":0,"ondisk_log_size":1581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1582,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":396,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.149955+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:16.149955+0000","last_peered":"2024-09-22T03:27:16.149955+0000","last_clean":"2024-09-22T03:27:16.149955+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:16.149955+0000","last_undegraded":"2024-09-22T03:27:16.149955+0000","last_fullsized":"2024-09-22T03:27:16.149955+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"198'1553","reported_seq":2091,"reported_epoch":199,"state":"peering","last_fresh":"2024-09-22T03:27:17.168568+0000","last_change":"2024-09-22T03:27:17.156905+0000","last_active":"2024-09-22T03:27:17.156905+0000","last_peered":"2024-09-22T03:27:16.877249+0000","last_clean":"2024-09-22T03:25:30.049680+0000","last_became_active":"2024-09-22T03:27:04.473583+0000","last_became_peered":"2024-09-22T03:27:04.473583+0000","last_unstale":"2024-09-22T03:27:17.168568+0000","last_undegraded":"2024-09-22T03:27:17.168568+0000","last_fullsized":"2024-09-22T03:27:17.168568+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1553,"log_dups_size":0,"ondisk_log_size":1553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48562176,"num_objects":741,"num_object_clones":0,"num_object_copies":2223,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1553,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.151562+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:16.151562+0000","last_peered":"2024-09-22T03:27:16.151562+0000","last_clean":"2024-09-22T03:27:16.151562+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:16.151562+0000","last_undegraded":"2024-09-22T03:27:16.151562+0000","last_fullsized":"2024-09-22T03:27:16.151562+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":761462808,"num_objects":11620,"num_object_clones":0,"num_object_copies":34860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":24428,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2662,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":142905344,"data_stored":2286288968,"data_compressed":108565229,"data_compressed_allocated":142893056,"data_compressed_original":2286288896,"omap_allocated":0,"internal_metadata":0},"log_size":24415,"ondisk_log_size":24415,"up":48,"acting":46,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101732,"kb_used_data":33428,"kb_used_omap":0,"kb_used_meta":68288,"kb_avail":94270108,"statfs":{"total":96636764160,"available":96532590592,"internally_reserved":0,"allocated":34230272,"data_stored":521890369,"data_compressed":24749736,"data_compressed_allocated":32575488,"data_compressed_original":521207808,"omap_allocated":0,"internal_metadata":69926912},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72336,"kb_used_data":34560,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94299504,"statfs":{"total":96636764160,"available":96562692096,"internally_reserved":0,"allocated":35389440,"data_stored":540109897,"data_compressed":25616835,"data_compressed_allocated":33718272,"data_compressed_original":539410432,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101412,"kb_used_data":39952,"kb_used_omap":0,"kb_used_meta":61440,"kb_avail":94270428,"statfs":{"total":96636764160,"available":96532918272,"internally_reserved":0,"allocated":40910848,"data_stored":628514625,"data_compressed":29814923,"data_compressed_allocated":39243776,"data_compressed_original":627818496,"omap_allocated":0,"internal_metadata":62914560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080606,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81592,"kb_used_data":40040,"kb_used_omap":0,"kb_used_meta":41536,"kb_avail":94290248,"statfs":{"total":96636764160,"available":96553213952,"internally_reserved":0,"allocated":41000960,"data_stored":630021934,"data_compressed":29886499,"data_compressed_allocated":39337984,"data_compressed_original":629325824,"omap_allocated":0,"internal_metadata":42532864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39063552,"data_stored":624951320,"data_compressed":29676031,"data_compressed_allocated":39059456,"data_compressed_original":624951296,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38662144,"data_stored":618528792,"data_compressed":29371055,"data_compressed_allocated":38658048,"data_compressed_original":618528768,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32759808,"data_stored":524156928,"data_compressed":24889775,"data_compressed_allocated":32759808,"data_compressed_original":524156928,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32419840,"data_stored":518651928,"data_compressed":24628368,"data_compressed_allocated":32415744,"data_compressed_original":518651904,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:19.493 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:27:19.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:19.620+0000 7f89a3317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:19.628+0000 7f89a3317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:19.667+0000 7f680f2b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:19.677+0000 7f680f2b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:19.945 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 454/35736 objects degraded (1.270%), 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":984,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":201,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975622,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+remapped","count":2},{"state_name":"peering","count":2}],"num_pgs":25,"num_pools":3,"num_objects":11912,"data_bytes":780927563,"bytes_used":394772480,"bytes_avail":386152284160,"bytes_total":386547056640,"inactive_pgs_ratio":0.079999998211860657,"degraded_objects":454,"degraded_total":35736,"degraded_ratio":0.012704275800313409,"recovering_objects_per_sec":160,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":642,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":81},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T03:27:01.429211+0000","services":{}},"progress_events":{}} 2024-09-22T03:27:19.957 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:27:19.958 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:20.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:20.056+0000 7efe86d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:20.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:20.063+0000 7efe86d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:20.255 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:20.256 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:20.275 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":683,"stamp":"2024-09-22T03:27:20.135657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":762053195,"num_objects":11624,"num_object_clones":0,"num_object_copies":34872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":24567,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2662,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24515,"ondisk_log_size":24515,"up":75,"acting":73,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":357072,"kb_used_data":147980,"kb_used_omap":0,"kb_used_meta":209024,"kb_avail":377130288,"statfs":{"total":386547056640,"available":386181414912,"internally_reserved":0,"allocated":151531520,"data_stored":2320536825,"data_compressed":110067993,"data_compressed_allocated":144875520,"data_compressed_original":2317762560,"omap_allocated":0,"internal_metadata":214040576},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-40173568,"num_objects":-613,"num_object_clones":0,"num_object_copies":-1839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":613,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.995255"},"pg_stats":[{"pgid":"6.f","version":"199'1534","reported_seq":2092,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.301309+0000","last_change":"2024-09-22T03:27:12.177019+0000","last_active":"2024-09-22T03:27:17.301309+0000","last_peered":"2024-09-22T03:27:17.301309+0000","last_clean":"2024-09-22T03:27:17.301309+0000","last_became_active":"2024-09-22T03:27:12.165305+0000","last_became_peered":"2024-09-22T03:27:12.165305+0000","last_unstale":"2024-09-22T03:27:17.301309+0000","last_undegraded":"2024-09-22T03:27:17.301309+0000","last_fullsized":"2024-09-22T03:27:17.301309+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1534,"log_dups_size":0,"ondisk_log_size":1534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47513600,"num_objects":725,"num_object_clones":0,"num_object_copies":2175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1535,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"198'1543","reported_seq":2124,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.723823+0000","last_change":"2024-09-22T03:27:13.172481+0000","last_active":"2024-09-22T03:27:16.723823+0000","last_peered":"2024-09-22T03:27:16.723823+0000","last_clean":"2024-09-22T03:27:16.723823+0000","last_became_active":"2024-09-22T03:27:13.160685+0000","last_became_peered":"2024-09-22T03:27:13.160685+0000","last_unstale":"2024-09-22T03:27:16.723823+0000","last_undegraded":"2024-09-22T03:27:16.723823+0000","last_fullsized":"2024-09-22T03:27:16.723823+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1543,"log_dups_size":0,"ondisk_log_size":1543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47579136,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1544,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"198'1477","reported_seq":1626,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.883830+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:16.883830+0000","last_peered":"2024-09-22T03:27:16.883830+0000","last_clean":"2024-09-22T03:27:16.883830+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:16.883830+0000","last_undegraded":"2024-09-22T03:27:16.883830+0000","last_fullsized":"2024-09-22T03:27:16.883830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1477,"log_dups_size":0,"ondisk_log_size":1477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45940760,"num_objects":702,"num_object_clones":0,"num_object_copies":2106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1478,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"199'1591","reported_seq":2164,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.525439+0000","last_change":"2024-09-22T03:27:14.173803+0000","last_active":"2024-09-22T03:27:17.525439+0000","last_peered":"2024-09-22T03:27:17.525439+0000","last_clean":"2024-09-22T03:27:17.525439+0000","last_became_active":"2024-09-22T03:27:14.167068+0000","last_became_peered":"2024-09-22T03:27:14.167068+0000","last_unstale":"2024-09-22T03:27:17.525439+0000","last_undegraded":"2024-09-22T03:27:17.525439+0000","last_fullsized":"2024-09-22T03:27:17.525439+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1591,"log_dups_size":0,"ondisk_log_size":1591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48365568,"num_objects":738,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1592,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"198'1559","reported_seq":2145,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.851952+0000","last_change":"2024-09-22T03:27:15.169223+0000","last_active":"2024-09-22T03:27:16.851952+0000","last_peered":"2024-09-22T03:27:16.851952+0000","last_clean":"2024-09-22T03:27:16.851952+0000","last_became_active":"2024-09-22T03:27:15.163663+0000","last_became_peered":"2024-09-22T03:27:15.163663+0000","last_unstale":"2024-09-22T03:27:16.851952+0000","last_undegraded":"2024-09-22T03:27:16.851952+0000","last_fullsized":"2024-09-22T03:27:16.851952+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1559,"log_dups_size":0,"ondisk_log_size":1559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47579136,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"199'1499","reported_seq":2071,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.313349+0000","last_change":"2024-09-22T03:27:16.205368+0000","last_active":"2024-09-22T03:27:17.313349+0000","last_peered":"2024-09-22T03:27:17.313349+0000","last_clean":"2024-09-22T03:27:17.313349+0000","last_became_active":"2024-09-22T03:27:16.194196+0000","last_became_peered":"2024-09-22T03:27:16.194196+0000","last_unstale":"2024-09-22T03:27:17.313349+0000","last_undegraded":"2024-09-22T03:27:17.313349+0000","last_fullsized":"2024-09-22T03:27:17.313349+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1499,"log_dups_size":0,"ondisk_log_size":1499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46661632,"num_objects":712,"num_object_clones":0,"num_object_copies":2136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1500,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"199'1511","reported_seq":1967,"reported_epoch":199,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:18.154332+0000","last_change":"2024-09-22T03:27:16.053905+0000","last_active":"2024-09-22T03:27:18.154332+0000","last_peered":"2024-09-22T03:27:18.154332+0000","last_clean":"2024-09-22T03:26:47.207978+0000","last_became_active":"2024-09-22T03:27:04.298212+0000","last_became_peered":"2024-09-22T03:27:04.298212+0000","last_unstale":"2024-09-22T03:27:18.154332+0000","last_undegraded":"2024-09-22T03:27:18.154332+0000","last_fullsized":"2024-09-22T03:27:04.158046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1511,"log_dups_size":0,"ondisk_log_size":1511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44761088,"num_objects":683,"num_object_clones":0,"num_object_copies":2049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1511,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":683}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"198'1448","reported_seq":1985,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.498382+0000","last_change":"2024-09-22T03:27:11.210627+0000","last_active":"2024-09-22T03:27:16.498382+0000","last_peered":"2024-09-22T03:27:16.498382+0000","last_clean":"2024-09-22T03:27:16.498382+0000","last_became_active":"2024-09-22T03:27:11.173122+0000","last_became_peered":"2024-09-22T03:27:11.173122+0000","last_unstale":"2024-09-22T03:27:16.498382+0000","last_undegraded":"2024-09-22T03:27:16.498382+0000","last_fullsized":"2024-09-22T03:27:16.498382+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1448,"log_dups_size":0,"ondisk_log_size":1448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45547520,"num_objects":695,"num_object_clones":0,"num_object_copies":2085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1449,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"198'1443","reported_seq":1561,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.509580+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:16.509580+0000","last_peered":"2024-09-22T03:27:16.509580+0000","last_clean":"2024-09-22T03:27:16.509580+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:16.509580+0000","last_undegraded":"2024-09-22T03:27:16.509580+0000","last_fullsized":"2024-09-22T03:27:16.509580+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1443,"log_dups_size":0,"ondisk_log_size":1443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47316992,"num_objects":722,"num_object_clones":0,"num_object_copies":2166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1444,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":370,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.158390+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:17.158390+0000","last_peered":"2024-09-22T03:27:17.158390+0000","last_clean":"2024-09-22T03:27:17.158390+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:17.158390+0000","last_undegraded":"2024-09-22T03:27:17.158390+0000","last_fullsized":"2024-09-22T03:27:17.158390+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"199'1555","reported_seq":1718,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.485205+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:17.485205+0000","last_peered":"2024-09-22T03:27:17.485205+0000","last_clean":"2024-09-22T03:27:17.485205+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:17.485205+0000","last_undegraded":"2024-09-22T03:27:17.485205+0000","last_fullsized":"2024-09-22T03:27:17.485205+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1555,"log_dups_size":0,"ondisk_log_size":1555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":768,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":473,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.147481+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:16.147481+0000","last_peered":"2024-09-22T03:27:16.147481+0000","last_clean":"2024-09-22T03:27:16.147481+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:16.147481+0000","last_undegraded":"2024-09-22T03:27:16.147481+0000","last_fullsized":"2024-09-22T03:27:16.147481+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"198'1513","reported_seq":1858,"reported_epoch":198,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:27:16.489343+0000","last_change":"2024-09-22T03:27:14.794805+0000","last_active":"2024-09-22T03:27:16.489343+0000","last_peered":"2024-09-22T03:27:16.489343+0000","last_clean":"2024-09-22T03:25:30.080890+0000","last_became_active":"2024-09-22T03:27:04.474157+0000","last_became_peered":"2024-09-22T03:27:04.474157+0000","last_unstale":"2024-09-22T03:27:16.489343+0000","last_undegraded":"2024-09-22T03:27:16.489343+0000","last_fullsized":"2024-09-22T03:27:04.261632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1513,"log_dups_size":0,"ondisk_log_size":1513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49545216,"num_objects":756,"num_object_clones":0,"num_object_copies":2268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1514,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":756}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":492,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.160890+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:17.160890+0000","last_peered":"2024-09-22T03:27:17.160890+0000","last_clean":"2024-09-22T03:27:17.160890+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:17.160890+0000","last_undegraded":"2024-09-22T03:27:17.160890+0000","last_fullsized":"2024-09-22T03:27:17.160890+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"199'1572","reported_seq":1741,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.331088+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:17.331088+0000","last_peered":"2024-09-22T03:27:17.331088+0000","last_clean":"2024-09-22T03:27:17.331088+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:17.331088+0000","last_undegraded":"2024-09-22T03:27:17.331088+0000","last_fullsized":"2024-09-22T03:27:17.331088+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1573,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":456,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.157448+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:17.157448+0000","last_peered":"2024-09-22T03:27:17.157448+0000","last_clean":"2024-09-22T03:27:17.157448+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:17.157448+0000","last_undegraded":"2024-09-22T03:27:17.157448+0000","last_fullsized":"2024-09-22T03:27:17.157448+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":591,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.154875+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:16.154875+0000","last_peered":"2024-09-22T03:27:16.154875+0000","last_clean":"2024-09-22T03:27:16.154875+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:16.154875+0000","last_undegraded":"2024-09-22T03:27:16.154875+0000","last_fullsized":"2024-09-22T03:27:16.154875+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"198'1525","reported_seq":1635,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.325471+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:16.325471+0000","last_peered":"2024-09-22T03:27:16.325471+0000","last_clean":"2024-09-22T03:27:16.325471+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:16.325471+0000","last_undegraded":"2024-09-22T03:27:16.325471+0000","last_fullsized":"2024-09-22T03:27:16.325471+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1525,"log_dups_size":0,"ondisk_log_size":1525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1526,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":266,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.148186+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:16.148186+0000","last_peered":"2024-09-22T03:27:16.148186+0000","last_clean":"2024-09-22T03:27:16.148186+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:16.148186+0000","last_undegraded":"2024-09-22T03:27:16.148186+0000","last_fullsized":"2024-09-22T03:27:16.148186+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"197'1511","reported_seq":2128,"reported_epoch":198,"state":"peering","last_fresh":"2024-09-22T03:27:16.036241+0000","last_change":"2024-09-22T03:27:16.151387+0000","last_active":"2024-09-22T03:27:16.151387+0000","last_peered":"2024-09-22T03:27:16.036241+0000","last_clean":"2024-09-22T03:25:30.070791+0000","last_became_active":"2024-09-22T03:27:04.472230+0000","last_became_peered":"2024-09-22T03:27:04.472230+0000","last_unstale":"2024-09-22T03:27:16.036241+0000","last_undegraded":"2024-09-22T03:27:16.036241+0000","last_fullsized":"2024-09-22T03:27:04.183417+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1511,"log_dups_size":0,"ondisk_log_size":1511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47251456,"num_objects":721,"num_object_clones":0,"num_object_copies":2163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1511,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":389,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.154219+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:17.154219+0000","last_peered":"2024-09-22T03:27:17.154219+0000","last_clean":"2024-09-22T03:27:17.154219+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:17.154219+0000","last_undegraded":"2024-09-22T03:27:17.154219+0000","last_fullsized":"2024-09-22T03:27:17.154219+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"199'1581","reported_seq":2199,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-22T03:27:17.330882+0000","last_change":"2024-09-22T03:27:10.180074+0000","last_active":"2024-09-22T03:27:17.330882+0000","last_peered":"2024-09-22T03:27:17.330882+0000","last_clean":"2024-09-22T03:27:17.330882+0000","last_became_active":"2024-09-22T03:27:10.168804+0000","last_became_peered":"2024-09-22T03:27:10.168804+0000","last_unstale":"2024-09-22T03:27:17.330882+0000","last_undegraded":"2024-09-22T03:27:17.330882+0000","last_fullsized":"2024-09-22T03:27:17.330882+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1581,"log_dups_size":0,"ondisk_log_size":1581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1582,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":396,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.149955+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:16.149955+0000","last_peered":"2024-09-22T03:27:16.149955+0000","last_clean":"2024-09-22T03:27:16.149955+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:16.149955+0000","last_undegraded":"2024-09-22T03:27:16.149955+0000","last_fullsized":"2024-09-22T03:27:16.149955+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"198'1553","reported_seq":2091,"reported_epoch":199,"state":"peering","last_fresh":"2024-09-22T03:27:17.168568+0000","last_change":"2024-09-22T03:27:17.156905+0000","last_active":"2024-09-22T03:27:17.156905+0000","last_peered":"2024-09-22T03:27:16.877249+0000","last_clean":"2024-09-22T03:25:30.049680+0000","last_became_active":"2024-09-22T03:27:04.473583+0000","last_became_peered":"2024-09-22T03:27:04.473583+0000","last_unstale":"2024-09-22T03:27:17.168568+0000","last_undegraded":"2024-09-22T03:27:17.168568+0000","last_fullsized":"2024-09-22T03:27:17.168568+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1553,"log_dups_size":0,"ondisk_log_size":1553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48562176,"num_objects":741,"num_object_clones":0,"num_object_copies":2223,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1553,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":343,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.151562+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:16.151562+0000","last_peered":"2024-09-22T03:27:16.151562+0000","last_clean":"2024-09-22T03:27:16.151562+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:16.151562+0000","last_undegraded":"2024-09-22T03:27:16.151562+0000","last_fullsized":"2024-09-22T03:27:16.151562+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":761462808,"num_objects":11620,"num_object_clones":0,"num_object_copies":34860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":24428,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2662,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":142905344,"data_stored":2286288968,"data_compressed":108565229,"data_compressed_allocated":142893056,"data_compressed_original":2286288896,"omap_allocated":0,"internal_metadata":0},"log_size":24415,"ondisk_log_size":24415,"up":48,"acting":46,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101732,"kb_used_data":33428,"kb_used_omap":0,"kb_used_meta":68288,"kb_avail":94270108,"statfs":{"total":96636764160,"available":96532590592,"internally_reserved":0,"allocated":34230272,"data_stored":521890369,"data_compressed":24749736,"data_compressed_allocated":32575488,"data_compressed_original":521207808,"omap_allocated":0,"internal_metadata":69926912},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72336,"kb_used_data":34560,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94299504,"statfs":{"total":96636764160,"available":96562692096,"internally_reserved":0,"allocated":35389440,"data_stored":540109897,"data_compressed":25616835,"data_compressed_allocated":33718272,"data_compressed_original":539410432,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101412,"kb_used_data":39952,"kb_used_omap":0,"kb_used_meta":61440,"kb_avail":94270428,"statfs":{"total":96636764160,"available":96532918272,"internally_reserved":0,"allocated":40910848,"data_stored":628514625,"data_compressed":29814923,"data_compressed_allocated":39243776,"data_compressed_original":627818496,"omap_allocated":0,"internal_metadata":62914560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080606,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81592,"kb_used_data":40040,"kb_used_omap":0,"kb_used_meta":41536,"kb_avail":94290248,"statfs":{"total":96636764160,"available":96553213952,"internally_reserved":0,"allocated":41000960,"data_stored":630021934,"data_compressed":29886499,"data_compressed_allocated":39337984,"data_compressed_original":629325824,"omap_allocated":0,"internal_metadata":42532864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39063552,"data_stored":624951320,"data_compressed":29676031,"data_compressed_allocated":39059456,"data_compressed_original":624951296,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38662144,"data_stored":618528792,"data_compressed":29371055,"data_compressed_allocated":38658048,"data_compressed_original":618528768,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32759808,"data_stored":524156928,"data_compressed":24889775,"data_compressed_allocated":32759808,"data_compressed_original":524156928,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32419840,"data_stored":518651928,"data_compressed":24628368,"data_compressed_allocated":32415744,"data_compressed_original":518651904,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:21.767 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:21.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:21.903+0000 7f314cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:21.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:21.911+0000 7f314cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:23.278 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:23.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:23.433+0000 7f83ef197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:23.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:23.445+0000 7f83ef197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:23.813 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:23.814 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:23.835 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":685,"stamp":"2024-09-22T03:27:23.433420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":691929675,"num_objects":10554,"num_object_clones":0,"num_object_copies":31662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":25637,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2857,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25582,"ondisk_log_size":25582,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":358124,"kb_used_data":136552,"kb_used_omap":0,"kb_used_meta":221504,"kb_avail":377129236,"statfs":{"total":386547056640,"available":386180337664,"internally_reserved":0,"allocated":139829248,"data_stored":2131511809,"data_compressed":101089873,"data_compressed_allocated":133058560,"data_compressed_original":2128691200,"omap_allocated":0,"internal_metadata":226820096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-88997888,"num_objects":-1358,"num_object_clones":0,"num_object_copies":-4074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1358,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000927"},"pg_stats":[{"pgid":"6.f","version":"202'1612","reported_seq":2179,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.783195+0000","last_change":"2024-09-22T03:27:12.177019+0000","last_active":"2024-09-22T03:27:22.783195+0000","last_peered":"2024-09-22T03:27:22.783195+0000","last_clean":"2024-09-22T03:27:22.783195+0000","last_became_active":"2024-09-22T03:27:12.165305+0000","last_became_peered":"2024-09-22T03:27:12.165305+0000","last_unstale":"2024-09-22T03:27:22.783195+0000","last_undegraded":"2024-09-22T03:27:22.783195+0000","last_fullsized":"2024-09-22T03:27:22.783195+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1612,"log_dups_size":0,"ondisk_log_size":1612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42401792,"num_objects":647,"num_object_clones":0,"num_object_copies":1941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1613,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"202'1603","reported_seq":2196,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.091643+0000","last_change":"2024-09-22T03:27:13.172481+0000","last_active":"2024-09-22T03:27:22.091643+0000","last_peered":"2024-09-22T03:27:22.091643+0000","last_clean":"2024-09-22T03:27:22.091643+0000","last_became_active":"2024-09-22T03:27:13.160685+0000","last_became_peered":"2024-09-22T03:27:13.160685+0000","last_unstale":"2024-09-22T03:27:22.091643+0000","last_undegraded":"2024-09-22T03:27:22.091643+0000","last_fullsized":"2024-09-22T03:27:22.091643+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1603,"log_dups_size":0,"ondisk_log_size":1603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43646976,"num_objects":666,"num_object_clones":0,"num_object_copies":1998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"202'1537","reported_seq":1698,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:21.922772+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:21.922772+0000","last_peered":"2024-09-22T03:27:21.922772+0000","last_clean":"2024-09-22T03:27:21.922772+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:21.922772+0000","last_undegraded":"2024-09-22T03:27:21.922772+0000","last_fullsized":"2024-09-22T03:27:21.922772+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1537,"log_dups_size":0,"ondisk_log_size":1537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42008600,"num_objects":642,"num_object_clones":0,"num_object_copies":1926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1538,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"202'1667","reported_seq":2249,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.958665+0000","last_change":"2024-09-22T03:27:14.173803+0000","last_active":"2024-09-22T03:27:22.958665+0000","last_peered":"2024-09-22T03:27:22.958665+0000","last_clean":"2024-09-22T03:27:22.958665+0000","last_became_active":"2024-09-22T03:27:14.167068+0000","last_became_peered":"2024-09-22T03:27:14.167068+0000","last_unstale":"2024-09-22T03:27:22.958665+0000","last_undegraded":"2024-09-22T03:27:22.958665+0000","last_fullsized":"2024-09-22T03:27:22.958665+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1667,"log_dups_size":0,"ondisk_log_size":1667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43384832,"num_objects":662,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1668,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"202'1614","reported_seq":2212,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.080376+0000","last_change":"2024-09-22T03:27:15.169223+0000","last_active":"2024-09-22T03:27:22.080376+0000","last_peered":"2024-09-22T03:27:22.080376+0000","last_clean":"2024-09-22T03:27:22.080376+0000","last_became_active":"2024-09-22T03:27:15.163663+0000","last_became_peered":"2024-09-22T03:27:15.163663+0000","last_unstale":"2024-09-22T03:27:22.080376+0000","last_undegraded":"2024-09-22T03:27:22.080376+0000","last_fullsized":"2024-09-22T03:27:22.080376+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1614,"log_dups_size":0,"ondisk_log_size":1614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43974656,"num_objects":671,"num_object_clones":0,"num_object_copies":2013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1615,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"202'1560","reported_seq":2141,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.847651+0000","last_change":"2024-09-22T03:27:16.205368+0000","last_active":"2024-09-22T03:27:22.847651+0000","last_peered":"2024-09-22T03:27:22.847651+0000","last_clean":"2024-09-22T03:27:22.847651+0000","last_became_active":"2024-09-22T03:27:16.194196+0000","last_became_peered":"2024-09-22T03:27:16.194196+0000","last_unstale":"2024-09-22T03:27:22.847651+0000","last_undegraded":"2024-09-22T03:27:22.847651+0000","last_fullsized":"2024-09-22T03:27:22.847651+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1560,"log_dups_size":0,"ondisk_log_size":1560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42663936,"num_objects":651,"num_object_clones":0,"num_object_copies":1953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1561,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"202'1589","reported_seq":2184,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:23.010821+0000","last_change":"2024-09-22T03:27:20.171570+0000","last_active":"2024-09-22T03:27:23.010821+0000","last_peered":"2024-09-22T03:27:23.010821+0000","last_clean":"2024-09-22T03:27:23.010821+0000","last_became_active":"2024-09-22T03:27:20.164248+0000","last_became_peered":"2024-09-22T03:27:20.164248+0000","last_unstale":"2024-09-22T03:27:23.010821+0000","last_undegraded":"2024-09-22T03:27:23.010821+0000","last_fullsized":"2024-09-22T03:27:23.010821+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1589,"log_dups_size":0,"ondisk_log_size":1589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39583744,"num_objects":604,"num_object_clones":0,"num_object_copies":1812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1590,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"202'1499","reported_seq":2048,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:21.482580+0000","last_change":"2024-09-22T03:27:11.210627+0000","last_active":"2024-09-22T03:27:21.482580+0000","last_peered":"2024-09-22T03:27:21.482580+0000","last_clean":"2024-09-22T03:27:21.482580+0000","last_became_active":"2024-09-22T03:27:11.173122+0000","last_became_peered":"2024-09-22T03:27:11.173122+0000","last_unstale":"2024-09-22T03:27:21.482580+0000","last_undegraded":"2024-09-22T03:27:21.482580+0000","last_fullsized":"2024-09-22T03:27:21.482580+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1499,"log_dups_size":0,"ondisk_log_size":1499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42205184,"num_objects":644,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1500,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"202'1498","reported_seq":1628,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:21.450615+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:21.450615+0000","last_peered":"2024-09-22T03:27:21.450615+0000","last_clean":"2024-09-22T03:27:21.450615+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:21.450615+0000","last_undegraded":"2024-09-22T03:27:21.450615+0000","last_fullsized":"2024-09-22T03:27:21.450615+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1498,"log_dups_size":0,"ondisk_log_size":1498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43712512,"num_objects":667,"num_object_clones":0,"num_object_copies":2001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":379,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.146981+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:20.146981+0000","last_peered":"2024-09-22T03:27:20.146981+0000","last_clean":"2024-09-22T03:27:20.146981+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:20.146981+0000","last_undegraded":"2024-09-22T03:27:20.146981+0000","last_fullsized":"2024-09-22T03:27:20.146981+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"202'1630","reported_seq":1802,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:23.092525+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:23.092525+0000","last_peered":"2024-09-22T03:27:23.092525+0000","last_clean":"2024-09-22T03:27:23.092525+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:23.092525+0000","last_undegraded":"2024-09-22T03:27:23.092525+0000","last_fullsized":"2024-09-22T03:27:23.092525+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1630,"log_dups_size":0,"ondisk_log_size":1630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45416448,"num_objects":693,"num_object_clones":0,"num_object_copies":2079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1631,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":485,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.143736+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:20.143736+0000","last_peered":"2024-09-22T03:27:20.143736+0000","last_clean":"2024-09-22T03:27:20.143736+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:20.143736+0000","last_undegraded":"2024-09-22T03:27:20.143736+0000","last_fullsized":"2024-09-22T03:27:20.143736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"202'1606","reported_seq":2083,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.956141+0000","last_change":"2024-09-22T03:27:19.186322+0000","last_active":"2024-09-22T03:27:22.956141+0000","last_peered":"2024-09-22T03:27:22.956141+0000","last_clean":"2024-09-22T03:27:22.956141+0000","last_became_active":"2024-09-22T03:27:19.180554+0000","last_became_peered":"2024-09-22T03:27:19.180554+0000","last_unstale":"2024-09-22T03:27:22.956141+0000","last_undegraded":"2024-09-22T03:27:22.956141+0000","last_fullsized":"2024-09-22T03:27:22.956141+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1606,"log_dups_size":0,"ondisk_log_size":1606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43450368,"num_objects":663,"num_object_clones":0,"num_object_copies":1989,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1607,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":501,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.145669+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:20.145669+0000","last_peered":"2024-09-22T03:27:20.145669+0000","last_clean":"2024-09-22T03:27:20.145669+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:20.145669+0000","last_undegraded":"2024-09-22T03:27:20.145669+0000","last_fullsized":"2024-09-22T03:27:20.145669+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"202'1623","reported_seq":1801,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:23.042894+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:23.042894+0000","last_peered":"2024-09-22T03:27:23.042894+0000","last_clean":"2024-09-22T03:27:23.042894+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:23.042894+0000","last_undegraded":"2024-09-22T03:27:23.042894+0000","last_fullsized":"2024-09-22T03:27:23.042894+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1623,"log_dups_size":0,"ondisk_log_size":1623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43384832,"num_objects":662,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1624,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":465,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.146114+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:20.146114+0000","last_peered":"2024-09-22T03:27:20.146114+0000","last_clean":"2024-09-22T03:27:20.146114+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:20.146114+0000","last_undegraded":"2024-09-22T03:27:20.146114+0000","last_fullsized":"2024-09-22T03:27:20.146114+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":603,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.146916+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:20.146916+0000","last_peered":"2024-09-22T03:27:20.146916+0000","last_clean":"2024-09-22T03:27:20.146916+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:20.146916+0000","last_undegraded":"2024-09-22T03:27:20.146916+0000","last_fullsized":"2024-09-22T03:27:20.146916+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"202'1579","reported_seq":1701,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:21.465159+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:21.465159+0000","last_peered":"2024-09-22T03:27:21.465159+0000","last_clean":"2024-09-22T03:27:21.465159+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:21.465159+0000","last_undegraded":"2024-09-22T03:27:21.465159+0000","last_fullsized":"2024-09-22T03:27:21.465159+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1579,"log_dups_size":0,"ondisk_log_size":1579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45350912,"num_objects":692,"num_object_clones":0,"num_object_copies":2076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1580,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":278,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.148282+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:20.148282+0000","last_peered":"2024-09-22T03:27:20.148282+0000","last_clean":"2024-09-22T03:27:20.148282+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:20.148282+0000","last_undegraded":"2024-09-22T03:27:20.148282+0000","last_fullsized":"2024-09-22T03:27:20.148282+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"202'1577","reported_seq":2212,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:21.996701+0000","last_change":"2024-09-22T03:27:17.185797+0000","last_active":"2024-09-22T03:27:21.996701+0000","last_peered":"2024-09-22T03:27:21.996701+0000","last_clean":"2024-09-22T03:27:21.996701+0000","last_became_active":"2024-09-22T03:27:17.174426+0000","last_became_peered":"2024-09-22T03:27:17.174426+0000","last_unstale":"2024-09-22T03:27:21.996701+0000","last_undegraded":"2024-09-22T03:27:21.996701+0000","last_fullsized":"2024-09-22T03:27:21.996701+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1577,"log_dups_size":0,"ondisk_log_size":1577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42860544,"num_objects":654,"num_object_clones":0,"num_object_copies":1962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1578,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":398,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.143383+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:20.143383+0000","last_peered":"2024-09-22T03:27:20.143383+0000","last_clean":"2024-09-22T03:27:20.143383+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:20.143383+0000","last_undegraded":"2024-09-22T03:27:20.143383+0000","last_fullsized":"2024-09-22T03:27:20.143383+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"202'1654","reported_seq":2281,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.898630+0000","last_change":"2024-09-22T03:27:10.180074+0000","last_active":"2024-09-22T03:27:22.898630+0000","last_peered":"2024-09-22T03:27:22.898630+0000","last_clean":"2024-09-22T03:27:22.898630+0000","last_became_active":"2024-09-22T03:27:10.168804+0000","last_became_peered":"2024-09-22T03:27:10.168804+0000","last_unstale":"2024-09-22T03:27:22.898630+0000","last_undegraded":"2024-09-22T03:27:22.898630+0000","last_fullsized":"2024-09-22T03:27:22.898630+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1654,"log_dups_size":0,"ondisk_log_size":1654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44105728,"num_objects":673,"num_object_clones":0,"num_object_copies":2019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1655,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":408,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.158163+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:20.158163+0000","last_peered":"2024-09-22T03:27:20.158163+0000","last_clean":"2024-09-22T03:27:20.158163+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:20.158163+0000","last_undegraded":"2024-09-22T03:27:20.158163+0000","last_fullsized":"2024-09-22T03:27:20.158163+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"202'1634","reported_seq":2187,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:22.928656+0000","last_change":"2024-09-22T03:27:18.200430+0000","last_active":"2024-09-22T03:27:22.928656+0000","last_peered":"2024-09-22T03:27:22.928656+0000","last_clean":"2024-09-22T03:27:22.928656+0000","last_became_active":"2024-09-22T03:27:18.193297+0000","last_became_peered":"2024-09-22T03:27:18.193297+0000","last_unstale":"2024-09-22T03:27:22.928656+0000","last_undegraded":"2024-09-22T03:27:22.928656+0000","last_fullsized":"2024-09-22T03:27:22.928656+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1634,"log_dups_size":0,"ondisk_log_size":1634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43188224,"num_objects":659,"num_object_clones":0,"num_object_copies":1977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1635,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":355,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.156702+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:20.156702+0000","last_peered":"2024-09-22T03:27:20.156702+0000","last_clean":"2024-09-22T03:27:20.156702+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:20.156702+0000","last_undegraded":"2024-09-22T03:27:20.156702+0000","last_fullsized":"2024-09-22T03:27:20.156702+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":691339288,"num_objects":10550,"num_object_clones":0,"num_object_copies":31650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":25498,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2857,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130134016,"data_stored":2081947720,"data_compressed":98862013,"data_compressed_allocated":130121728,"data_compressed_original":2081947648,"omap_allocated":0,"internal_metadata":0},"log_size":25482,"ondisk_log_size":25482,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103232,"kb_used_data":30768,"kb_used_omap":0,"kb_used_meta":72448,"kb_avail":94268608,"statfs":{"total":96636764160,"available":96531054592,"internally_reserved":0,"allocated":31506432,"data_stored":477925619,"data_compressed":22661584,"data_compressed_allocated":29827072,"data_compressed_original":477233152,"omap_allocated":0,"internal_metadata":74186752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949780,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72876,"kb_used_data":30936,"kb_used_omap":0,"kb_used_meta":41920,"kb_avail":94298964,"statfs":{"total":96636764160,"available":96562139136,"internally_reserved":0,"allocated":31678464,"data_stored":480350971,"data_compressed":22778691,"data_compressed_allocated":29982720,"data_compressed_original":479641600,"omap_allocated":0,"internal_metadata":42926080},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103000,"kb_used_data":37384,"kb_used_omap":0,"kb_used_meta":65600,"kb_avail":94268840,"statfs":{"total":96636764160,"available":96531292160,"internally_reserved":0,"allocated":38281216,"data_stored":585929491,"data_compressed":27792123,"data_compressed_allocated":36581376,"data_compressed_original":585220096,"omap_allocated":0,"internal_metadata":67174400},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79016,"kb_used_data":37464,"kb_used_omap":0,"kb_used_meta":41536,"kb_avail":94292824,"statfs":{"total":96636764160,"available":96555851776,"internally_reserved":0,"allocated":38363136,"data_stored":587305728,"data_compressed":27857475,"data_compressed_allocated":36667392,"data_compressed_original":586596352,"omap_allocated":0,"internal_metadata":42532864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36294656,"data_stored":580648984,"data_compressed":27572319,"data_compressed_allocated":36290560,"data_compressed_original":580648960,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35643392,"data_stored":570228760,"data_compressed":27077511,"data_compressed_allocated":35639296,"data_compressed_original":570228736,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29110272,"data_stored":465764352,"data_compressed":22116983,"data_compressed_allocated":29110272,"data_compressed_original":465764352,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29085696,"data_stored":465305624,"data_compressed":22095200,"data_compressed_allocated":29081600,"data_compressed_original":465305600,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:23.836 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:27:23.837 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:27:23.837 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:27:23.837 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:27:23.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:23.961+0000 7f88f9117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:23.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:23.974+0000 7f88f9117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:27:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:27:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:27:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:27:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:27:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:27:24.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:27:24.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:27:24.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:27:24.104 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:27:24.104 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:27:24.106 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:24.107 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:24.107 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:27:24.130 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:24.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.228+0000 7f456f14e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.249+0000 7f456f14e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:27:24.276 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:27:24.277 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:27:24.277 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:27:24.277 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:27:24.277 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:27:24.284 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:24.285 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:24.285 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:27:24.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.287+0000 7fb566797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.297+0000 7fb566797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.444+0000 7f0f7cd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.457+0000 7f0f7cd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.571 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:27:24.572 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:27:24.575 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:24.576 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:24.576 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:27:24.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.753+0000 7f6f2e26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:24.765+0000 7f6f2e26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:24.809 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:24.809 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:27:24.809 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:27:24.810 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:27:24.813 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:24.814 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:26.669 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:26.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:26.811+0000 7f16dea6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:26.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:26.821+0000 7f16dea6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:29.057 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:29.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:29.186+0000 7f1bde197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:29.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:29.192+0000 7f1bde197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:29.815 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 1, 0, 2] 2024-09-22T03:27:29.815 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:27:29.815 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [3, 1, 0, 2] 2024-09-22T03:27:29.816 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T03:27:29.816 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:27:31.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-22T03:27:31.358 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:31.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:31.471+0000 7f4611597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:31.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:31.475+0000 7f4611597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:35.916 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T03:27:36.509 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-22T03:27:40.917 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [3, 1, 2] 2024-09-22T03:27:40.917 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:27:40.917 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:41.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:41.030+0000 7ff3a9717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:41.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:41.037+0000 7ff3a9717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:41.212 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:27:41.212 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:27:41.225 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":693,"stamp":"2024-09-22T03:27:39.436896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548536907,"num_objects":8366,"num_object_clones":0,"num_object_copies":25098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":27825,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2857,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27770,"ondisk_log_size":27770,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":326000,"kb_used_data":108592,"kb_used_omap":0,"kb_used_meta":217344,"kb_avail":377161360,"statfs":{"total":386547056640,"available":386213232640,"internally_reserved":0,"allocated":111198208,"data_stored":1673415169,"data_compressed":79336993,"data_compressed_allocated":104427520,"data_compressed_original":1670594560,"omap_allocated":0,"internal_metadata":222560256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-95813632,"num_objects":-1462,"num_object_clones":0,"num_object_copies":-4386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1462,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002768"},"pg_stats":[{"pgid":"6.f","version":"202'1747","reported_seq":2314,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.817933+0000","last_change":"2024-09-22T03:27:12.177019+0000","last_active":"2024-09-22T03:27:29.817933+0000","last_peered":"2024-09-22T03:27:29.817933+0000","last_clean":"2024-09-22T03:27:29.817933+0000","last_became_active":"2024-09-22T03:27:12.165305+0000","last_became_peered":"2024-09-22T03:27:12.165305+0000","last_unstale":"2024-09-22T03:27:29.817933+0000","last_undegraded":"2024-09-22T03:27:29.817933+0000","last_fullsized":"2024-09-22T03:27:29.817933+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1747,"log_dups_size":0,"ondisk_log_size":1747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":512,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1748,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"202'1761","reported_seq":2354,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.816866+0000","last_change":"2024-09-22T03:27:13.172481+0000","last_active":"2024-09-22T03:27:29.816866+0000","last_peered":"2024-09-22T03:27:29.816866+0000","last_clean":"2024-09-22T03:27:29.816866+0000","last_became_active":"2024-09-22T03:27:13.160685+0000","last_became_peered":"2024-09-22T03:27:13.160685+0000","last_unstale":"2024-09-22T03:27:29.816866+0000","last_undegraded":"2024-09-22T03:27:29.816866+0000","last_fullsized":"2024-09-22T03:27:29.816866+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33292288,"num_objects":508,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1762,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"202'1685","reported_seq":1846,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.642086+0000","last_change":"2024-09-22T03:26:47.560504+0000","last_active":"2024-09-22T03:27:29.642086+0000","last_peered":"2024-09-22T03:27:29.642086+0000","last_clean":"2024-09-22T03:27:29.642086+0000","last_became_active":"2024-09-22T03:26:47.272312+0000","last_became_peered":"2024-09-22T03:26:47.272312+0000","last_unstale":"2024-09-22T03:27:29.642086+0000","last_undegraded":"2024-09-22T03:27:29.642086+0000","last_fullsized":"2024-09-22T03:27:29.642086+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1685,"log_dups_size":0,"ondisk_log_size":1685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32309272,"num_objects":494,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1686,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"202'1800","reported_seq":2382,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.777969+0000","last_change":"2024-09-22T03:27:14.173803+0000","last_active":"2024-09-22T03:27:29.777969+0000","last_peered":"2024-09-22T03:27:29.777969+0000","last_clean":"2024-09-22T03:27:29.777969+0000","last_became_active":"2024-09-22T03:27:14.167068+0000","last_became_peered":"2024-09-22T03:27:14.167068+0000","last_unstale":"2024-09-22T03:27:29.777969+0000","last_undegraded":"2024-09-22T03:27:29.777969+0000","last_fullsized":"2024-09-22T03:27:29.777969+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1800,"log_dups_size":0,"ondisk_log_size":1800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":529,"num_object_clones":0,"num_object_copies":1587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1801,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"202'1778","reported_seq":2376,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.773243+0000","last_change":"2024-09-22T03:27:15.169223+0000","last_active":"2024-09-22T03:27:29.773243+0000","last_peered":"2024-09-22T03:27:29.773243+0000","last_clean":"2024-09-22T03:27:29.773243+0000","last_became_active":"2024-09-22T03:27:15.163663+0000","last_became_peered":"2024-09-22T03:27:15.163663+0000","last_unstale":"2024-09-22T03:27:29.773243+0000","last_undegraded":"2024-09-22T03:27:29.773243+0000","last_fullsized":"2024-09-22T03:27:29.773243+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1778,"log_dups_size":0,"ondisk_log_size":1778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33226752,"num_objects":507,"num_object_clones":0,"num_object_copies":1521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1779,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"202'1725","reported_seq":2306,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.795675+0000","last_change":"2024-09-22T03:27:16.205368+0000","last_active":"2024-09-22T03:27:29.795675+0000","last_peered":"2024-09-22T03:27:29.795675+0000","last_clean":"2024-09-22T03:27:29.795675+0000","last_became_active":"2024-09-22T03:27:16.194196+0000","last_became_peered":"2024-09-22T03:27:16.194196+0000","last_unstale":"2024-09-22T03:27:29.795675+0000","last_undegraded":"2024-09-22T03:27:29.795675+0000","last_fullsized":"2024-09-22T03:27:29.795675+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1725,"log_dups_size":0,"ondisk_log_size":1725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1726,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"202'1712","reported_seq":2307,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.719694+0000","last_change":"2024-09-22T03:27:20.171570+0000","last_active":"2024-09-22T03:27:29.719694+0000","last_peered":"2024-09-22T03:27:29.719694+0000","last_clean":"2024-09-22T03:27:29.719694+0000","last_became_active":"2024-09-22T03:27:20.164248+0000","last_became_peered":"2024-09-22T03:27:20.164248+0000","last_unstale":"2024-09-22T03:27:29.719694+0000","last_undegraded":"2024-09-22T03:27:29.719694+0000","last_fullsized":"2024-09-22T03:27:29.719694+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1712,"log_dups_size":0,"ondisk_log_size":1712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31522816,"num_objects":481,"num_object_clones":0,"num_object_copies":1443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1713,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"202'1601","reported_seq":2150,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:26.422901+0000","last_change":"2024-09-22T03:27:11.210627+0000","last_active":"2024-09-22T03:27:26.422901+0000","last_peered":"2024-09-22T03:27:26.422901+0000","last_clean":"2024-09-22T03:27:26.422901+0000","last_became_active":"2024-09-22T03:27:11.173122+0000","last_became_peered":"2024-09-22T03:27:11.173122+0000","last_unstale":"2024-09-22T03:27:26.422901+0000","last_undegraded":"2024-09-22T03:27:26.422901+0000","last_fullsized":"2024-09-22T03:27:26.422901+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1601,"log_dups_size":0,"ondisk_log_size":1601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35520512,"num_objects":542,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1602,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"202'1607","reported_seq":1737,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:26.463194+0000","last_change":"2024-09-22T03:26:47.511397+0000","last_active":"2024-09-22T03:27:26.463194+0000","last_peered":"2024-09-22T03:27:26.463194+0000","last_clean":"2024-09-22T03:27:26.463194+0000","last_became_active":"2024-09-22T03:26:47.268767+0000","last_became_peered":"2024-09-22T03:26:47.268767+0000","last_unstale":"2024-09-22T03:27:26.463194+0000","last_undegraded":"2024-09-22T03:27:26.463194+0000","last_fullsized":"2024-09-22T03:27:26.463194+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1607,"log_dups_size":0,"ondisk_log_size":1607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36569088,"num_objects":558,"num_object_clones":0,"num_object_copies":1674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1608,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":379,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.146981+0000","last_change":"2024-09-22T03:27:04.187341+0000","last_active":"2024-09-22T03:27:20.146981+0000","last_peered":"2024-09-22T03:27:20.146981+0000","last_clean":"2024-09-22T03:27:20.146981+0000","last_became_active":"2024-09-22T03:27:03.204208+0000","last_became_peered":"2024-09-22T03:27:03.204208+0000","last_unstale":"2024-09-22T03:27:20.146981+0000","last_undegraded":"2024-09-22T03:27:20.146981+0000","last_fullsized":"2024-09-22T03:27:20.146981+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"202'1760","reported_seq":1932,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.721779+0000","last_change":"2024-09-22T03:26:47.737675+0000","last_active":"2024-09-22T03:27:29.721779+0000","last_peered":"2024-09-22T03:27:29.721779+0000","last_clean":"2024-09-22T03:27:29.721779+0000","last_became_active":"2024-09-22T03:26:47.274188+0000","last_became_peered":"2024-09-22T03:26:47.274188+0000","last_unstale":"2024-09-22T03:27:29.721779+0000","last_undegraded":"2024-09-22T03:27:29.721779+0000","last_fullsized":"2024-09-22T03:27:29.721779+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1760,"log_dups_size":0,"ondisk_log_size":1760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36896768,"num_objects":563,"num_object_clones":0,"num_object_copies":1689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1761,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":485,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.143736+0000","last_change":"2024-09-22T03:27:04.264864+0000","last_active":"2024-09-22T03:27:20.143736+0000","last_peered":"2024-09-22T03:27:20.143736+0000","last_clean":"2024-09-22T03:27:20.143736+0000","last_became_active":"2024-09-22T03:27:03.221624+0000","last_became_peered":"2024-09-22T03:27:03.221624+0000","last_unstale":"2024-09-22T03:27:20.143736+0000","last_undegraded":"2024-09-22T03:27:20.143736+0000","last_fullsized":"2024-09-22T03:27:20.143736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"202'1756","reported_seq":2233,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.824229+0000","last_change":"2024-09-22T03:27:19.186322+0000","last_active":"2024-09-22T03:27:29.824229+0000","last_peered":"2024-09-22T03:27:29.824229+0000","last_clean":"2024-09-22T03:27:29.824229+0000","last_became_active":"2024-09-22T03:27:19.180554+0000","last_became_peered":"2024-09-22T03:27:19.180554+0000","last_unstale":"2024-09-22T03:27:29.824229+0000","last_undegraded":"2024-09-22T03:27:29.824229+0000","last_fullsized":"2024-09-22T03:27:29.824229+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1756,"log_dups_size":0,"ondisk_log_size":1756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":513,"num_object_clones":0,"num_object_copies":1539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1757,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":501,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.145669+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:27:20.145669+0000","last_peered":"2024-09-22T03:27:20.145669+0000","last_clean":"2024-09-22T03:27:20.145669+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:27:20.145669+0000","last_undegraded":"2024-09-22T03:27:20.145669+0000","last_fullsized":"2024-09-22T03:27:20.145669+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"202'1761","reported_seq":1939,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.752942+0000","last_change":"2024-09-22T03:26:47.650573+0000","last_active":"2024-09-22T03:27:29.752942+0000","last_peered":"2024-09-22T03:27:29.752942+0000","last_clean":"2024-09-22T03:27:29.752942+0000","last_became_active":"2024-09-22T03:26:47.259802+0000","last_became_peered":"2024-09-22T03:26:47.259802+0000","last_unstale":"2024-09-22T03:27:29.752942+0000","last_undegraded":"2024-09-22T03:27:29.752942+0000","last_fullsized":"2024-09-22T03:27:29.752942+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":524,"num_object_clones":0,"num_object_copies":1572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1762,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":465,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.146114+0000","last_change":"2024-09-22T03:26:48.206674+0000","last_active":"2024-09-22T03:27:20.146114+0000","last_peered":"2024-09-22T03:27:20.146114+0000","last_clean":"2024-09-22T03:27:20.146114+0000","last_became_active":"2024-09-22T03:26:47.270597+0000","last_became_peered":"2024-09-22T03:26:47.270597+0000","last_unstale":"2024-09-22T03:27:20.146114+0000","last_undegraded":"2024-09-22T03:27:20.146114+0000","last_fullsized":"2024-09-22T03:27:20.146114+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":603,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.146916+0000","last_change":"2024-09-22T03:27:03.228777+0000","last_active":"2024-09-22T03:27:20.146916+0000","last_peered":"2024-09-22T03:27:20.146916+0000","last_clean":"2024-09-22T03:27:20.146916+0000","last_became_active":"2024-09-22T03:27:03.225489+0000","last_became_peered":"2024-09-22T03:27:03.225489+0000","last_unstale":"2024-09-22T03:27:20.146916+0000","last_undegraded":"2024-09-22T03:27:20.146916+0000","last_fullsized":"2024-09-22T03:27:20.146916+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"202'1695","reported_seq":1817,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:26.521696+0000","last_change":"2024-09-22T03:26:47.499248+0000","last_active":"2024-09-22T03:27:26.521696+0000","last_peered":"2024-09-22T03:27:26.521696+0000","last_clean":"2024-09-22T03:27:26.521696+0000","last_became_active":"2024-09-22T03:26:47.267687+0000","last_became_peered":"2024-09-22T03:26:47.267687+0000","last_unstale":"2024-09-22T03:27:26.521696+0000","last_undegraded":"2024-09-22T03:27:26.521696+0000","last_fullsized":"2024-09-22T03:27:26.521696+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1695,"log_dups_size":0,"ondisk_log_size":1695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":576,"num_object_clones":0,"num_object_copies":1728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1696,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":278,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.148282+0000","last_change":"2024-09-22T03:27:04.286290+0000","last_active":"2024-09-22T03:27:20.148282+0000","last_peered":"2024-09-22T03:27:20.148282+0000","last_clean":"2024-09-22T03:27:20.148282+0000","last_became_active":"2024-09-22T03:27:03.266791+0000","last_became_peered":"2024-09-22T03:27:03.266791+0000","last_unstale":"2024-09-22T03:27:20.148282+0000","last_undegraded":"2024-09-22T03:27:20.148282+0000","last_fullsized":"2024-09-22T03:27:20.148282+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"202'1738","reported_seq":2373,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.791938+0000","last_change":"2024-09-22T03:27:17.185797+0000","last_active":"2024-09-22T03:27:29.791938+0000","last_peered":"2024-09-22T03:27:29.791938+0000","last_clean":"2024-09-22T03:27:29.791938+0000","last_became_active":"2024-09-22T03:27:17.174426+0000","last_became_peered":"2024-09-22T03:27:17.174426+0000","last_unstale":"2024-09-22T03:27:29.791938+0000","last_undegraded":"2024-09-22T03:27:29.791938+0000","last_fullsized":"2024-09-22T03:27:29.791938+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1738,"log_dups_size":0,"ondisk_log_size":1738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32309248,"num_objects":493,"num_object_clones":0,"num_object_copies":1479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1739,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":398,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.143383+0000","last_change":"2024-09-22T03:27:04.160685+0000","last_active":"2024-09-22T03:27:20.143383+0000","last_peered":"2024-09-22T03:27:20.143383+0000","last_clean":"2024-09-22T03:27:20.143383+0000","last_became_active":"2024-09-22T03:27:03.265979+0000","last_became_peered":"2024-09-22T03:27:03.265979+0000","last_unstale":"2024-09-22T03:27:20.143383+0000","last_undegraded":"2024-09-22T03:27:20.143383+0000","last_fullsized":"2024-09-22T03:27:20.143383+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"202'1793","reported_seq":2420,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.802226+0000","last_change":"2024-09-22T03:27:10.180074+0000","last_active":"2024-09-22T03:27:29.802226+0000","last_peered":"2024-09-22T03:27:29.802226+0000","last_clean":"2024-09-22T03:27:29.802226+0000","last_became_active":"2024-09-22T03:27:10.168804+0000","last_became_peered":"2024-09-22T03:27:10.168804+0000","last_unstale":"2024-09-22T03:27:29.802226+0000","last_undegraded":"2024-09-22T03:27:29.802226+0000","last_fullsized":"2024-09-22T03:27:29.802226+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1793,"log_dups_size":0,"ondisk_log_size":1793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34996224,"num_objects":534,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1794,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":408,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.158163+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:27:20.158163+0000","last_peered":"2024-09-22T03:27:20.158163+0000","last_clean":"2024-09-22T03:27:20.158163+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:27:20.158163+0000","last_undegraded":"2024-09-22T03:27:20.158163+0000","last_fullsized":"2024-09-22T03:27:20.158163+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"202'1751","reported_seq":2304,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:29.642760+0000","last_change":"2024-09-22T03:27:18.200430+0000","last_active":"2024-09-22T03:27:29.642760+0000","last_peered":"2024-09-22T03:27:29.642760+0000","last_clean":"2024-09-22T03:27:29.642760+0000","last_became_active":"2024-09-22T03:27:18.193297+0000","last_became_peered":"2024-09-22T03:27:18.193297+0000","last_unstale":"2024-09-22T03:27:29.642760+0000","last_undegraded":"2024-09-22T03:27:29.642760+0000","last_fullsized":"2024-09-22T03:27:29.642760+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1751,"log_dups_size":0,"ondisk_log_size":1751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35520512,"num_objects":542,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1752,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":355,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-22T03:27:20.156702+0000","last_change":"2024-09-22T03:26:48.218703+0000","last_active":"2024-09-22T03:27:20.156702+0000","last_peered":"2024-09-22T03:27:20.156702+0000","last_clean":"2024-09-22T03:27:20.156702+0000","last_became_active":"2024-09-22T03:26:47.268572+0000","last_became_peered":"2024-09-22T03:26:47.268572+0000","last_unstale":"2024-09-22T03:27:20.156702+0000","last_undegraded":"2024-09-22T03:27:20.156702+0000","last_fullsized":"2024-09-22T03:27:20.156702+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":547946520,"num_objects":8362,"num_object_clones":0,"num_object_copies":25086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27686,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2857,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104042496,"data_stored":1664483400,"data_compressed":79038573,"data_compressed_allocated":104030208,"data_compressed_original":1664483328,"omap_allocated":0,"internal_metadata":0},"log_size":27670,"ondisk_log_size":27670,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100804,"kb_used_data":24180,"kb_used_omap":0,"kb_used_meta":76608,"kb_avail":94271036,"statfs":{"total":96636764160,"available":96533540864,"internally_reserved":0,"allocated":24760320,"data_stored":369987827,"data_compressed":17536120,"data_compressed_allocated":23080960,"data_compressed_original":369295360,"omap_allocated":0,"internal_metadata":78446592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949795,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57632,"kb_used_data":24016,"kb_used_omap":0,"kb_used_meta":33600,"kb_avail":94314208,"statfs":{"total":96636764160,"available":96577748992,"internally_reserved":0,"allocated":24592384,"data_stored":366973691,"data_compressed":17394931,"data_compressed_allocated":22896640,"data_compressed_original":366264320,"omap_allocated":0,"internal_metadata":34406400},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85548,"kb_used_data":28252,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94286292,"statfs":{"total":96636764160,"available":96549163008,"internally_reserved":0,"allocated":28930048,"data_stored":436310803,"data_compressed":20687427,"data_compressed_allocated":27230208,"data_compressed_original":435601408,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":181,"seq":777389080616,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82016,"kb_used_data":32144,"kb_used_omap":0,"kb_used_meta":49856,"kb_avail":94289824,"statfs":{"total":96636764160,"available":96552779776,"internally_reserved":0,"allocated":32915456,"data_stored":500142848,"data_compressed":23718515,"data_compressed_allocated":31219712,"data_compressed_original":499433472,"omap_allocated":0,"internal_metadata":51052544},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30908416,"data_stored":494469144,"data_compressed":23480039,"data_compressed_allocated":30904320,"data_compressed_original":494469120,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27193344,"data_stored":435027992,"data_compressed":20657455,"data_compressed_allocated":27189248,"data_compressed_original":435027968,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22855680,"data_stored":365690880,"data_compressed":17364959,"data_compressed_allocated":22855680,"data_compressed_original":365690880,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23085056,"data_stored":369295384,"data_compressed":17536120,"data_compressed_allocated":23080960,"data_compressed_original":369295360,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:46.227 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [3, 1, 2] 2024-09-22T03:27:46.228 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:27:46.228 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-22T03:27:46.228 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:27:46.228 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:27:46.231 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:27:46.231 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:27:46.341 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:27:46.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:46.345 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:27:51.346 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:27:51.444 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:27:51.446 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:51.447 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:27:55.143 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:27:55.293 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:27:55.294 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:27:56.448 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:27:56.551 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:27:56.551 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [] 2024-09-22T03:27:56.551 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:27:57.294 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:57.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:57.423+0000 7f940aaba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:57.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:57.429+0000 7f940aaba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:59.692 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:59.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:59.829+0000 7f0a90597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:59.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:27:59.835+0000 7f0a90597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.560 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 1, 2, 0] 2024-09-22T03:28:01.560 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:28:01.560 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:28:01.561 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:28:01.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:01.698+0000 7f824ad57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:01.715+0000 7f824ad57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.828 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:01.828 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:28:01.828 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:28:01.828 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:28:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:28:01.832 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:01.832 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:01.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:28:01.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:01.972+0000 7f6dcfd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:01.984+0000 7f6dcfd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:02.012 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:28:02.013 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:28:02.014 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:28:02.018 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:02.019 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:02.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:28:02.170 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:02.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:02.171+0000 7f0a3dab2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:02.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:02.190+0000 7f0a3dab2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:02.232 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:28:02.233 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:28:02.237 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:02.237 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:02.238 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:28:02.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:02.323+0000 7fdcf0618640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:02.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:02.330+0000 7fdcf0618640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:02.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:02.428+0000 7fe256058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:02.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:02.438+0000 7fe256058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:02.457 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:02.457 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:28:02.457 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:28:02.457 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:28:02.457 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:28:02.458 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:28:02.458 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:28:02.458 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:28:02.458 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:28:02.458 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:28:02.458 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:28:02.458 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:28:02.460 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:02.460 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:04.622 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:04.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:04.766+0000 7f81a84c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:04.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:04.775+0000 7f81a84c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:07.038 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:07.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:07.189+0000 7f25d686c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:07.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:07.197+0000 7f25d686c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:07.462 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 1, 2, 0] 2024-09-22T03:28:07.462 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:28:07.462 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [3, 1, 2, 0] 2024-09-22T03:28:07.462 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:28:07.463 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:28:09.372 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:09.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:09.486+0000 7f0911e18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:09.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:09.491+0000 7f0911e18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:09.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:28:13.563 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:28:15.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:28:18.564 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [3, 2, 0] 2024-09-22T03:28:18.564 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:28:18.565 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:18.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:18.679+0000 7f249b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:18.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:18.685+0000 7f249b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:18.863 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:18.864 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:28:18.876 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":716,"stamp":"2024-09-22T03:28:17.445068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329450059,"num_objects":5023,"num_object_clones":0,"num_object_copies":15069,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":31168,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2859,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31113,"ondisk_log_size":31113,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":290260,"kb_used_data":67920,"kb_used_omap":0,"kb_used_meta":222272,"kb_avail":377197100,"statfs":{"total":386547056640,"available":386249830400,"internally_reserved":0,"allocated":69550080,"data_stored":1005004777,"data_compressed":47594596,"data_compressed_allocated":62648320,"data_compressed_original":1002127360,"omap_allocated":0,"internal_metadata":227606528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-97386496,"num_objects":-1486,"num_object_clones":0,"num_object_copies":-4458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1486,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002572"},"pg_stats":[{"pgid":"6.f","version":"206'1946","reported_seq":2535,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.400667+0000","last_change":"2024-09-22T03:27:57.473119+0000","last_active":"2024-09-22T03:28:07.400667+0000","last_peered":"2024-09-22T03:28:07.400667+0000","last_clean":"2024-09-22T03:28:07.400667+0000","last_became_active":"2024-09-22T03:27:57.320411+0000","last_became_peered":"2024-09-22T03:27:57.320411+0000","last_unstale":"2024-09-22T03:28:07.400667+0000","last_undegraded":"2024-09-22T03:28:07.400667+0000","last_fullsized":"2024-09-22T03:28:07.400667+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1946,"log_dups_size":0,"ondisk_log_size":1946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":313,"num_object_clones":0,"num_object_copies":939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1947,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"206'1940","reported_seq":2556,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.029476+0000","last_change":"2024-09-22T03:27:57.485523+0000","last_active":"2024-09-22T03:28:07.029476+0000","last_peered":"2024-09-22T03:28:07.029476+0000","last_clean":"2024-09-22T03:28:07.029476+0000","last_became_active":"2024-09-22T03:27:57.303565+0000","last_became_peered":"2024-09-22T03:27:57.303565+0000","last_unstale":"2024-09-22T03:28:07.029476+0000","last_undegraded":"2024-09-22T03:28:07.029476+0000","last_fullsized":"2024-09-22T03:28:07.029476+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1940,"log_dups_size":0,"ondisk_log_size":1940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":329,"num_object_clones":0,"num_object_copies":987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1941,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"206'1870","reported_seq":2044,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.038649+0000","last_change":"2024-09-22T03:27:57.314365+0000","last_active":"2024-09-22T03:28:07.038649+0000","last_peered":"2024-09-22T03:28:07.038649+0000","last_clean":"2024-09-22T03:28:07.038649+0000","last_became_active":"2024-09-22T03:27:57.297094+0000","last_became_peered":"2024-09-22T03:27:57.297094+0000","last_unstale":"2024-09-22T03:28:07.038649+0000","last_undegraded":"2024-09-22T03:28:07.038649+0000","last_fullsized":"2024-09-22T03:28:07.038649+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1870,"log_dups_size":0,"ondisk_log_size":1870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185112,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1871,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"206'2008","reported_seq":2602,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.472063+0000","last_change":"2024-09-22T03:27:14.173803+0000","last_active":"2024-09-22T03:28:07.472063+0000","last_peered":"2024-09-22T03:28:07.472063+0000","last_clean":"2024-09-22T03:28:07.472063+0000","last_became_active":"2024-09-22T03:27:14.167068+0000","last_became_peered":"2024-09-22T03:27:14.167068+0000","last_unstale":"2024-09-22T03:28:07.472063+0000","last_undegraded":"2024-09-22T03:28:07.472063+0000","last_fullsized":"2024-09-22T03:28:07.472063+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2008,"log_dups_size":0,"ondisk_log_size":2008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037056,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2009,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"206'1959","reported_seq":2569,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.029073+0000","last_change":"2024-09-22T03:27:15.169223+0000","last_active":"2024-09-22T03:28:07.029073+0000","last_peered":"2024-09-22T03:28:07.029073+0000","last_clean":"2024-09-22T03:28:07.029073+0000","last_became_active":"2024-09-22T03:27:15.163663+0000","last_became_peered":"2024-09-22T03:27:15.163663+0000","last_unstale":"2024-09-22T03:28:07.029073+0000","last_undegraded":"2024-09-22T03:28:07.029073+0000","last_fullsized":"2024-09-22T03:28:07.029073+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1959,"log_dups_size":0,"ondisk_log_size":1959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1960,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"206'1922","reported_seq":2522,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.475124+0000","last_change":"2024-09-22T03:27:57.426276+0000","last_active":"2024-09-22T03:28:07.475124+0000","last_peered":"2024-09-22T03:28:07.475124+0000","last_clean":"2024-09-22T03:28:07.475124+0000","last_became_active":"2024-09-22T03:27:57.290616+0000","last_became_peered":"2024-09-22T03:27:57.290616+0000","last_unstale":"2024-09-22T03:28:07.475124+0000","last_undegraded":"2024-09-22T03:28:07.475124+0000","last_fullsized":"2024-09-22T03:28:07.475124+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1922,"log_dups_size":0,"ondisk_log_size":1922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":289,"num_object_clones":0,"num_object_copies":867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1923,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"206'1882","reported_seq":2489,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.444998+0000","last_change":"2024-09-22T03:27:20.171570+0000","last_active":"2024-09-22T03:28:07.444998+0000","last_peered":"2024-09-22T03:28:07.444998+0000","last_clean":"2024-09-22T03:28:07.444998+0000","last_became_active":"2024-09-22T03:27:20.164248+0000","last_became_peered":"2024-09-22T03:27:20.164248+0000","last_unstale":"2024-09-22T03:28:07.444998+0000","last_undegraded":"2024-09-22T03:28:07.444998+0000","last_fullsized":"2024-09-22T03:28:07.444998+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1882,"log_dups_size":0,"ondisk_log_size":1882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1883,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"206'1858","reported_seq":2419,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.313136+0000","last_change":"2024-09-22T03:27:57.323000+0000","last_active":"2024-09-22T03:28:07.313136+0000","last_peered":"2024-09-22T03:28:07.313136+0000","last_clean":"2024-09-22T03:28:07.313136+0000","last_became_active":"2024-09-22T03:27:57.315204+0000","last_became_peered":"2024-09-22T03:27:57.315204+0000","last_unstale":"2024-09-22T03:28:07.313136+0000","last_undegraded":"2024-09-22T03:28:07.313136+0000","last_fullsized":"2024-09-22T03:28:07.313136+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1858,"log_dups_size":0,"ondisk_log_size":1858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":285,"num_object_clones":0,"num_object_copies":855,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1859,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"206'1854","reported_seq":1996,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.412971+0000","last_change":"2024-09-22T03:27:57.340876+0000","last_active":"2024-09-22T03:28:07.412971+0000","last_peered":"2024-09-22T03:28:07.412971+0000","last_clean":"2024-09-22T03:28:07.412971+0000","last_became_active":"2024-09-22T03:27:57.336096+0000","last_became_peered":"2024-09-22T03:27:57.336096+0000","last_unstale":"2024-09-22T03:28:07.412971+0000","last_undegraded":"2024-09-22T03:28:07.412971+0000","last_fullsized":"2024-09-22T03:28:07.412971+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1854,"log_dups_size":0,"ondisk_log_size":1854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1855,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":393,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.159426+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:28:00.159426+0000","last_peered":"2024-09-22T03:28:00.159426+0000","last_clean":"2024-09-22T03:28:00.159426+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:28:00.159426+0000","last_undegraded":"2024-09-22T03:28:00.159426+0000","last_fullsized":"2024-09-22T03:28:00.159426+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"206'1997","reported_seq":2182,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.466642+0000","last_change":"2024-09-22T03:27:57.249920+0000","last_active":"2024-09-22T03:28:07.466642+0000","last_peered":"2024-09-22T03:28:07.466642+0000","last_clean":"2024-09-22T03:28:07.466642+0000","last_became_active":"2024-09-22T03:27:57.245832+0000","last_became_peered":"2024-09-22T03:27:57.245832+0000","last_unstale":"2024-09-22T03:28:07.466642+0000","last_undegraded":"2024-09-22T03:28:07.466642+0000","last_fullsized":"2024-09-22T03:28:07.466642+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1997,"log_dups_size":0,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1998,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":449,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.167460+0000","last_change":"2024-09-22T03:28:00.174365+0000","last_active":"2024-09-22T03:28:00.167460+0000","last_peered":"2024-09-22T03:28:00.167460+0000","last_clean":"2024-09-22T03:28:00.167460+0000","last_became_active":"2024-09-22T03:27:57.328295+0000","last_became_peered":"2024-09-22T03:27:57.328295+0000","last_unstale":"2024-09-22T03:28:00.167460+0000","last_undegraded":"2024-09-22T03:28:00.167460+0000","last_fullsized":"2024-09-22T03:28:00.167460+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"206'1966","reported_seq":2465,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.491262+0000","last_change":"2024-09-22T03:27:57.489568+0000","last_active":"2024-09-22T03:28:07.491262+0000","last_peered":"2024-09-22T03:28:07.491262+0000","last_clean":"2024-09-22T03:28:07.491262+0000","last_became_active":"2024-09-22T03:27:57.314117+0000","last_became_peered":"2024-09-22T03:27:57.314117+0000","last_unstale":"2024-09-22T03:28:07.491262+0000","last_undegraded":"2024-09-22T03:28:07.491262+0000","last_fullsized":"2024-09-22T03:28:07.491262+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1966,"log_dups_size":0,"ondisk_log_size":1966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1967,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":513,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.175018+0000","last_change":"2024-09-22T03:27:04.163151+0000","last_active":"2024-09-22T03:28:00.175018+0000","last_peered":"2024-09-22T03:28:00.175018+0000","last_clean":"2024-09-22T03:28:00.175018+0000","last_became_active":"2024-09-22T03:27:03.275365+0000","last_became_peered":"2024-09-22T03:27:03.275365+0000","last_unstale":"2024-09-22T03:28:00.175018+0000","last_undegraded":"2024-09-22T03:28:00.175018+0000","last_fullsized":"2024-09-22T03:28:00.175018+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"206'1977","reported_seq":2174,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.483671+0000","last_change":"2024-09-22T03:27:57.311495+0000","last_active":"2024-09-22T03:28:07.483671+0000","last_peered":"2024-09-22T03:28:07.483671+0000","last_clean":"2024-09-22T03:28:07.483671+0000","last_became_active":"2024-09-22T03:27:57.236369+0000","last_became_peered":"2024-09-22T03:27:57.236369+0000","last_unstale":"2024-09-22T03:28:07.483671+0000","last_undegraded":"2024-09-22T03:28:07.483671+0000","last_fullsized":"2024-09-22T03:28:07.483671+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1977,"log_dups_size":0,"ondisk_log_size":1977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1978,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":479,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.158891+0000","last_change":"2024-09-22T03:28:00.178184+0000","last_active":"2024-09-22T03:28:00.158891+0000","last_peered":"2024-09-22T03:28:00.158891+0000","last_clean":"2024-09-22T03:28:00.158891+0000","last_became_active":"2024-09-22T03:27:57.236791+0000","last_became_peered":"2024-09-22T03:27:57.236791+0000","last_unstale":"2024-09-22T03:28:00.158891+0000","last_undegraded":"2024-09-22T03:28:00.158891+0000","last_fullsized":"2024-09-22T03:28:00.158891+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":567,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.171291+0000","last_change":"2024-09-22T03:27:57.355202+0000","last_active":"2024-09-22T03:28:00.171291+0000","last_peered":"2024-09-22T03:28:00.171291+0000","last_clean":"2024-09-22T03:28:00.171291+0000","last_became_active":"2024-09-22T03:27:57.347834+0000","last_became_peered":"2024-09-22T03:27:57.347834+0000","last_unstale":"2024-09-22T03:28:00.171291+0000","last_undegraded":"2024-09-22T03:28:00.171291+0000","last_fullsized":"2024-09-22T03:28:00.171291+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"206'1941","reported_seq":2075,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.348929+0000","last_change":"2024-09-22T03:27:57.327220+0000","last_active":"2024-09-22T03:28:07.348929+0000","last_peered":"2024-09-22T03:28:07.348929+0000","last_clean":"2024-09-22T03:28:07.348929+0000","last_became_active":"2024-09-22T03:27:57.321605+0000","last_became_peered":"2024-09-22T03:27:57.321605+0000","last_unstale":"2024-09-22T03:28:07.348929+0000","last_undegraded":"2024-09-22T03:28:07.348929+0000","last_fullsized":"2024-09-22T03:28:07.348929+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1941,"log_dups_size":0,"ondisk_log_size":1941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21626880,"num_objects":330,"num_object_clones":0,"num_object_copies":990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1942,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.187909+0000","last_change":"2024-09-22T03:28:00.190442+0000","last_active":"2024-09-22T03:28:00.187909+0000","last_peered":"2024-09-22T03:28:00.187909+0000","last_clean":"2024-09-22T03:28:00.187909+0000","last_became_active":"2024-09-22T03:27:57.292247+0000","last_became_peered":"2024-09-22T03:27:57.292247+0000","last_unstale":"2024-09-22T03:28:00.187909+0000","last_undegraded":"2024-09-22T03:28:00.187909+0000","last_fullsized":"2024-09-22T03:28:00.187909+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"206'1922","reported_seq":2576,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.101495+0000","last_change":"2024-09-22T03:27:57.409255+0000","last_active":"2024-09-22T03:28:07.101495+0000","last_peered":"2024-09-22T03:28:07.101495+0000","last_clean":"2024-09-22T03:28:07.101495+0000","last_became_active":"2024-09-22T03:27:57.287726+0000","last_became_peered":"2024-09-22T03:27:57.287726+0000","last_unstale":"2024-09-22T03:28:07.101495+0000","last_undegraded":"2024-09-22T03:28:07.101495+0000","last_fullsized":"2024-09-22T03:28:07.101495+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1922,"log_dups_size":0,"ondisk_log_size":1922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1923,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":412,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.173939+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:28:00.173939+0000","last_peered":"2024-09-22T03:28:00.173939+0000","last_clean":"2024-09-22T03:28:00.173939+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:28:00.173939+0000","last_undegraded":"2024-09-22T03:28:00.173939+0000","last_fullsized":"2024-09-22T03:28:00.173939+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"206'1999","reported_seq":2648,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.452234+0000","last_change":"2024-09-22T03:27:57.538527+0000","last_active":"2024-09-22T03:28:07.452234+0000","last_peered":"2024-09-22T03:28:07.452234+0000","last_clean":"2024-09-22T03:28:07.452234+0000","last_became_active":"2024-09-22T03:27:57.336387+0000","last_became_peered":"2024-09-22T03:27:57.336387+0000","last_unstale":"2024-09-22T03:28:07.452234+0000","last_undegraded":"2024-09-22T03:28:07.452234+0000","last_fullsized":"2024-09-22T03:28:07.452234+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1999,"log_dups_size":0,"ondisk_log_size":1999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21495808,"num_objects":328,"num_object_clones":0,"num_object_copies":984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2000,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":420,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.199654+0000","last_change":"2024-09-22T03:27:04.224259+0000","last_active":"2024-09-22T03:28:00.199654+0000","last_peered":"2024-09-22T03:28:00.199654+0000","last_clean":"2024-09-22T03:28:00.199654+0000","last_became_active":"2024-09-22T03:27:03.202217+0000","last_became_peered":"2024-09-22T03:27:03.202217+0000","last_unstale":"2024-09-22T03:28:00.199654+0000","last_undegraded":"2024-09-22T03:28:00.199654+0000","last_fullsized":"2024-09-22T03:28:00.199654+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"206'1972","reported_seq":2538,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:07.432585+0000","last_change":"2024-09-22T03:27:57.341822+0000","last_active":"2024-09-22T03:28:07.432585+0000","last_peered":"2024-09-22T03:28:07.432585+0000","last_clean":"2024-09-22T03:28:07.432585+0000","last_became_active":"2024-09-22T03:27:57.326249+0000","last_became_peered":"2024-09-22T03:27:57.326249+0000","last_unstale":"2024-09-22T03:28:07.432585+0000","last_undegraded":"2024-09-22T03:28:07.432585+0000","last_fullsized":"2024-09-22T03:28:07.432585+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":1972,"log_dups_size":0,"ondisk_log_size":1972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037056,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1973,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":369,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.199000+0000","last_change":"2024-09-22T03:28:00.207074+0000","last_active":"2024-09-22T03:28:00.199000+0000","last_peered":"2024-09-22T03:28:00.199000+0000","last_clean":"2024-09-22T03:28:00.199000+0000","last_became_active":"2024-09-22T03:27:57.272391+0000","last_became_peered":"2024-09-22T03:27:57.272391+0000","last_unstale":"2024-09-22T03:28:00.199000+0000","last_undegraded":"2024-09-22T03:28:00.199000+0000","last_fullsized":"2024-09-22T03:28:00.199000+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":328859672,"num_objects":5019,"num_object_clones":0,"num_object_copies":15057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":31029,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2859,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61734912,"data_stored":987562056,"data_compressed":46894728,"data_compressed_allocated":61722624,"data_compressed_original":987561984,"omap_allocated":0,"internal_metadata":0},"log_size":31013,"ondisk_log_size":31013,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83708,"kb_used_data":15404,"kb_used_omap":0,"kb_used_meta":68288,"kb_avail":94288132,"statfs":{"total":96636764160,"available":96551047168,"internally_reserved":0,"allocated":15773696,"data_stored":225691757,"data_compressed":10683496,"data_compressed_allocated":14061568,"data_compressed_original":224985088,"omap_allocated":0,"internal_metadata":69926912},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949830,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57340,"kb_used_data":15404,"kb_used_omap":0,"kb_used_meta":41920,"kb_avail":94314500,"statfs":{"total":96636764160,"available":96578048000,"internally_reserved":0,"allocated":15773696,"data_stored":225364597,"data_compressed":10669900,"data_compressed_allocated":14045184,"data_compressed_original":224641024,"omap_allocated":0,"internal_metadata":42926080},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":159,"seq":682899800309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":84840,"kb_used_data":19220,"kb_used_omap":0,"kb_used_meta":65600,"kb_avail":94287000,"statfs":{"total":96636764160,"available":96549888000,"internally_reserved":0,"allocated":19681280,"data_stored":287820429,"data_compressed":13635636,"data_compressed_allocated":17948672,"data_compressed_original":287096832,"omap_allocated":0,"internal_metadata":67174400},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328404,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64372,"kb_used_data":17892,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94307468,"statfs":{"total":96636764160,"available":96570847232,"internally_reserved":0,"allocated":18321408,"data_stored":266127994,"data_compressed":12605564,"data_compressed_allocated":16592896,"data_compressed_original":265404416,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16556032,"data_stored":264831000,"data_compressed":12575592,"data_compressed_allocated":16551936,"data_compressed_original":264830976,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":273678360,"data_compressed":12995712,"data_compressed_allocated":17104896,"data_compressed_original":273678336,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14004224,"data_stored":224067584,"data_compressed":10639928,"data_compressed_allocated":14004224,"data_compressed_original":224067584,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14065664,"data_stored":224985112,"data_compressed":10683496,"data_compressed_allocated":14061568,"data_compressed_original":224985088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:23.878 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [3, 2, 0] 2024-09-22T03:28:23.878 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:28:23.878 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-22T03:28:23.879 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:28:23.879 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:28:23.882 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:28:23.882 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:28:23.992 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:28:23.995 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:23.995 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:28:28.996 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:28:29.104 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:28:29.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:29.108 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:28:32.152 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:28:32.255 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:28:32.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:28:34.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:28:34.243 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:28:34.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:34.247 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:28:34.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:34.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:34.376+0000 7f26be858640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:34.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:34.382+0000 7f26be858640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:34.433 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:28:34.621 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:28:34.622 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:28:36.622 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:36.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:36.769+0000 7f898526c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:36.776+0000 7f898526c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.891 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:28:36.994 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:28:36.994 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:28:38.995 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:39.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:39.134+0000 7f4b83757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:39.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:39.141+0000 7f4b83757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:39.248 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:28:39.295 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-22T03:28:39.396 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:28:39.397 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:28:39.397 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:28:39.400 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:39.400 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-22T03:28:41.398 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:41.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:41.500+0000 7f80be06c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:41.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:41.505+0000 7f80be06c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.573 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:43.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:43.719+0000 7fc17a357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:43.726+0000 7fc17a357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:44.402 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "background_recovery_sub", 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554680, 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": {} 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.565 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 39, 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.293659+0000" 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.293670+0000" 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.566 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.312619+0000" 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.567 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.312771+0000" 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.568 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.312830+0000" 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.569 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.234458+0000" 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.313276+0000" 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.570 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.313937+0000" 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.571 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.314270+0000" 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.572 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.573 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.574 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.575 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.576 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.577 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "replicated_request", 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554679, 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "RepRequest": { 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "reqid": "client.5276.0:33845", 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "6.1", 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 211, 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "min_epoch": 210, 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "oid": "6:8fa46c67:::benchmark_data_smithi146_48150_object15819:head", 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2" 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.578 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "replicated_request", 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554681, 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "RepRequest": { 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "reqid": "client.5276.0:33850", 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "6.1", 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 211, 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "min_epoch": 210, 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "oid": "6:82715623:::benchmark_data_smithi146_48150_object15824:head", 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "2" 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.579 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "replicated_request", 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 201, 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "RepRequest": { 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "reqid": "client.5276.0:33849", 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "6.6", 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 211, 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "min_epoch": 210, 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "oid": "6:6fdfcf79:::benchmark_data_smithi146_48150_object15823:head", 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "3" 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.580 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "background_recovery_sub", 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 33554677, 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": {} 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 37, 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.291212+0000" 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.581 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.291220+0000" 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.291605+0000" 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.582 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.312315+0000" 2024-09-22T03:28:44.583 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.314365+0000" 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.584 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.244813+0000" 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.316086+0000" 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.585 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.586 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.317188+0000" 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.317688+0000" 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.587 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.588 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.589 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.590 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.591 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.592 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 41, 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.294911+0000" 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.593 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.294918+0000" 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.312905+0000" 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.594 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.313024+0000" 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.595 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.317037+0000" 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.596 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.244898+0000" 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.317588+0000" 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.597 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.319302+0000" 2024-09-22T03:28:44.598 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.599 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.600 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.601 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.602 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.603 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.604 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 50, 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.309198+0000" 2024-09-22T03:28:44.605 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.309207+0000" 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.315004+0000" 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.606 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.316398+0000" 2024-09-22T03:28:44.607 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.320045+0000" 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.608 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.244965+0000" 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.320762+0000" 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.609 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.610 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.611 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.611 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.611 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.612 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.613 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.614 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.615 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.616 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.617 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.618 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.618 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.618 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 35, 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.288730+0000" 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.619 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.288742+0000" 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.288882+0000" 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.620 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.289128+0000" 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.312204+0000" 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.313199+0000" 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.622 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.252519+0000" 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.314209+0000" 2024-09-22T03:28:44.623 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.624 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.314682+0000" 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.315593+0000" 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.625 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.626 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.627 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.628 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.629 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.630 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 53, 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.631 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.322280+0000" 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.322293+0000" 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.324107+0000" 2024-09-22T03:28:44.632 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.633 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.325216+0000" 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.328724+0000" 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.634 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.260516+0000" 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.635 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.636 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.639 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.640 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.641 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.642 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "replicated_request", 2024-09-22T03:28:44.643 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 203, 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "RepRequest": { 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "reqid": "client.5276.0:33858", 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "pgid": "6.4", 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 211, 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "min_epoch": 210, 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "oid": "6:20633dd5:::benchmark_data_smithi146_48150_object15832:head", 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "from": "3" 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 43, 2024-09-22T03:28:44.644 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.296641+0000" 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.296647+0000" 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.645 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.314190+0000" 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.646 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.315055+0000" 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.318041+0000" 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.647 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.251835+0000" 2024-09-22T03:28:44.648 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.318472+0000" 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.318669+0000" 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.318834+0000" 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.651 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.652 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.654 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.655 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.656 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 55, 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.334356+0000" 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.334377+0000" 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.657 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.335752+0000" 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.658 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.336777+0000" 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:28:44.659 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.338238+0000" 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.660 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:42.251883+0000" 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.338489+0000" 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:28:44.661 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:28:41.338608+0000" 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.662 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.663 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.664 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.666 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.668 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:28:44.669 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:28:45.895 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:46.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:46.045+0000 7fdd7df97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:46.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:46.051+0000 7fdd7df97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:48.555 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:48.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:48.670+0000 7f0d7fa18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:48.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:48.675+0000 7f0d7fa18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:49.670 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 2, 0, 1] 2024-09-22T03:28:49.671 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:28:49.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:49.809+0000 7fe06b797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:49.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:49.821+0000 7fe06b797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:50.148 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:28:50.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:28:50.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:50.306+0000 7f51e76d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:50.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:50.320+0000 7f51e76d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:50.968 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:51.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:51.096+0000 7f45a3b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:51.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:51.101+0000 7f45a3b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:51.149 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:28:51.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:28:51.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:51.300+0000 7f82b7263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:51.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:51.313+0000 7f82b7263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:52.528 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:28:52.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:28:52.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:52.682+0000 7f91b8157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:52.696+0000 7f91b8157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:53.215 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:53.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:53.348+0000 7f77b8117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:53.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:53.355+0000 7f77b8117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:53.532 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:28:53.553 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:28:53.553 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:53.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:53.710+0000 7fd1d2717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:53.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:53.721+0000 7fd1d2717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.018 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:54.019 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:28:54.038 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":746,"stamp":"2024-09-22T03:28:53.533060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32703051,"num_objects":495,"num_object_clones":0,"num_object_copies":1485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":35696,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3724,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35645,"ondisk_log_size":35645,"up":75,"acting":65,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":242004,"kb_used_data":20948,"kb_used_omap":0,"kb_used_meta":220992,"kb_avail":377245356,"statfs":{"total":386547056640,"available":386299244544,"internally_reserved":0,"allocated":21450752,"data_stored":230319377,"data_compressed":10801420,"data_compressed_allocated":14221312,"data_compressed_original":227295232,"omap_allocated":0,"internal_metadata":226295808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-59965440,"num_objects":-915,"num_object_clones":0,"num_object_copies":-2745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":915,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010047"},"pg_stats":[{"pgid":"6.f","version":"217'2234","reported_seq":3180,"reported_epoch":217,"state":"active+recovery_wait","last_fresh":"2024-09-22T03:28:52.828389+0000","last_change":"2024-09-22T03:28:49.630613+0000","last_active":"2024-09-22T03:28:52.828389+0000","last_peered":"2024-09-22T03:28:52.828389+0000","last_clean":"2024-09-22T03:28:07.444665+0000","last_became_active":"2024-09-22T03:28:49.595919+0000","last_became_peered":"2024-09-22T03:28:49.595919+0000","last_unstale":"2024-09-22T03:28:52.828389+0000","last_undegraded":"2024-09-22T03:28:52.828389+0000","last_fullsized":"2024-09-22T03:28:52.828389+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2235,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"0,1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"216'2210","reported_seq":3188,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.321730+0000","last_change":"2024-09-22T03:28:51.207954+0000","last_active":"2024-09-22T03:28:51.321730+0000","last_peered":"2024-09-22T03:28:51.321730+0000","last_clean":"2024-09-22T03:28:51.321730+0000","last_became_active":"2024-09-22T03:28:51.187017+0000","last_became_peered":"2024-09-22T03:28:51.187017+0000","last_unstale":"2024-09-22T03:28:51.321730+0000","last_undegraded":"2024-09-22T03:28:51.321730+0000","last_fullsized":"2024-09-22T03:28:51.321730+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2210,"log_dups_size":0,"ondisk_log_size":2210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2211,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"216'2138","reported_seq":2614,"reported_epoch":216,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:51.887493+0000","last_change":"2024-09-22T03:28:48.040111+0000","last_active":"2024-09-22T03:28:51.887493+0000","last_peered":"2024-09-22T03:28:51.887493+0000","last_clean":"2024-09-22T03:28:07.418415+0000","last_became_active":"2024-09-22T03:28:43.449859+0000","last_became_peered":"2024-09-22T03:28:43.449859+0000","last_unstale":"2024-09-22T03:28:51.887493+0000","last_undegraded":"2024-09-22T03:28:51.887493+0000","last_fullsized":"2024-09-22T03:28:43.266518+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2687000,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2138,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.c","version":"217'2313","reported_seq":3014,"reported_epoch":217,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:53.150846+0000","last_change":"2024-09-22T03:28:52.285232+0000","last_active":"2024-09-22T03:28:53.150846+0000","last_peered":"2024-09-22T03:28:53.150846+0000","last_clean":"2024-09-22T03:28:07.483127+0000","last_became_active":"2024-09-22T03:28:43.555328+0000","last_became_peered":"2024-09-22T03:28:43.555328+0000","last_unstale":"2024-09-22T03:28:53.150846+0000","last_undegraded":"2024-09-22T03:28:53.150846+0000","last_fullsized":"2024-09-22T03:28:43.259727+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2313,"log_dups_size":0,"ondisk_log_size":2313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":17}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"217'2258","reported_seq":2899,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:52.966321+0000","last_change":"2024-09-22T03:28:43.568261+0000","last_active":"2024-09-22T03:28:52.966321+0000","last_peered":"2024-09-22T03:28:52.966321+0000","last_clean":"2024-09-22T03:28:07.431554+0000","last_became_active":"2024-09-22T03:28:43.562050+0000","last_became_peered":"2024-09-22T03:28:43.562050+0000","last_unstale":"2024-09-22T03:28:52.966321+0000","last_undegraded":"2024-09-22T03:28:43.414054+0000","last_fullsized":"2024-09-22T03:28:43.387740+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2259,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"217'2186","reported_seq":3134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.805946+0000","last_change":"2024-09-22T03:28:50.193603+0000","last_active":"2024-09-22T03:28:52.805946+0000","last_peered":"2024-09-22T03:28:52.805946+0000","last_clean":"2024-09-22T03:28:52.805946+0000","last_became_active":"2024-09-22T03:28:50.182409+0000","last_became_peered":"2024-09-22T03:28:50.182409+0000","last_unstale":"2024-09-22T03:28:52.805946+0000","last_undegraded":"2024-09-22T03:28:52.805946+0000","last_fullsized":"2024-09-22T03:28:52.805946+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2187,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"217'2179","reported_seq":3042,"reported_epoch":217,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:53.142528+0000","last_change":"2024-09-22T03:28:50.020409+0000","last_active":"2024-09-22T03:28:53.142528+0000","last_peered":"2024-09-22T03:28:53.142528+0000","last_clean":"2024-09-22T03:28:07.457799+0000","last_became_active":"2024-09-22T03:28:43.547696+0000","last_became_peered":"2024-09-22T03:28:43.547696+0000","last_unstale":"2024-09-22T03:28:53.142528+0000","last_undegraded":"2024-09-22T03:28:53.142528+0000","last_fullsized":"2024-09-22T03:28:43.229591+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2179,"log_dups_size":0,"ondisk_log_size":2179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":15}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"216'2100","reported_seq":2691,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.661009+0000","last_change":"2024-09-22T03:27:57.323000+0000","last_active":"2024-09-22T03:28:51.661009+0000","last_peered":"2024-09-22T03:28:51.661009+0000","last_clean":"2024-09-22T03:28:51.661009+0000","last_became_active":"2024-09-22T03:27:57.315204+0000","last_became_peered":"2024-09-22T03:27:57.315204+0000","last_unstale":"2024-09-22T03:28:51.661009+0000","last_undegraded":"2024-09-22T03:28:51.661009+0000","last_fullsized":"2024-09-22T03:28:51.661009+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2100,"log_dups_size":0,"ondisk_log_size":2100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2101,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"216'2125","reported_seq":2310,"reported_epoch":216,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:51.698393+0000","last_change":"2024-09-22T03:28:43.750347+0000","last_active":"2024-09-22T03:28:51.698393+0000","last_peered":"2024-09-22T03:28:51.698393+0000","last_clean":"2024-09-22T03:28:07.505453+0000","last_became_active":"2024-09-22T03:28:43.459458+0000","last_became_peered":"2024-09-22T03:28:43.459458+0000","last_unstale":"2024-09-22T03:28:51.698393+0000","last_undegraded":"2024-09-22T03:28:43.744545+0000","last_fullsized":"2024-09-22T03:28:43.299387+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2125,"log_dups_size":0,"ondisk_log_size":2125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2126,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":426,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.571925+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:28:52.571925+0000","last_peered":"2024-09-22T03:28:52.571925+0000","last_clean":"2024-09-22T03:28:52.571925+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:28:52.571925+0000","last_undegraded":"2024-09-22T03:28:52.571925+0000","last_fullsized":"2024-09-22T03:28:52.571925+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"217'2305","reported_seq":2528,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:53.048381+0000","last_change":"2024-09-22T03:28:43.663585+0000","last_active":"2024-09-22T03:28:53.048381+0000","last_peered":"2024-09-22T03:28:53.048381+0000","last_clean":"2024-09-22T03:28:07.512449+0000","last_became_active":"2024-09-22T03:28:43.415468+0000","last_became_peered":"2024-09-22T03:28:43.415468+0000","last_unstale":"2024-09-22T03:28:53.048381+0000","last_undegraded":"2024-09-22T03:28:43.660936+0000","last_fullsized":"2024-09-22T03:28:43.261374+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":484,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.168452+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:28:51.168452+0000","last_peered":"2024-09-22T03:28:51.168452+0000","last_clean":"2024-09-22T03:28:51.168452+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:28:51.168452+0000","last_undegraded":"2024-09-22T03:28:51.168452+0000","last_fullsized":"2024-09-22T03:28:51.168452+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"216'2244","reported_seq":2776,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.540578+0000","last_change":"2024-09-22T03:27:57.489568+0000","last_active":"2024-09-22T03:28:52.540578+0000","last_peered":"2024-09-22T03:28:52.540578+0000","last_clean":"2024-09-22T03:28:52.540578+0000","last_became_active":"2024-09-22T03:27:57.314117+0000","last_became_peered":"2024-09-22T03:27:57.314117+0000","last_unstale":"2024-09-22T03:28:52.540578+0000","last_undegraded":"2024-09-22T03:28:52.540578+0000","last_fullsized":"2024-09-22T03:28:52.540578+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2244,"log_dups_size":0,"ondisk_log_size":2244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":551,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.544931+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:28:52.544931+0000","last_peered":"2024-09-22T03:28:52.544931+0000","last_clean":"2024-09-22T03:28:52.544931+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:28:52.544931+0000","last_undegraded":"2024-09-22T03:28:52.544931+0000","last_fullsized":"2024-09-22T03:28:52.544931+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"217'2263","reported_seq":2516,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:53.152011+0000","last_change":"2024-09-22T03:28:43.598307+0000","last_active":"2024-09-22T03:28:53.152011+0000","last_peered":"2024-09-22T03:28:53.152011+0000","last_clean":"2024-09-22T03:28:07.488866+0000","last_became_active":"2024-09-22T03:28:43.420961+0000","last_became_peered":"2024-09-22T03:28:43.420961+0000","last_unstale":"2024-09-22T03:28:53.152011+0000","last_undegraded":"2024-09-22T03:28:43.596998+0000","last_fullsized":"2024-09-22T03:28:43.260012+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2264,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":517,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.571486+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:28:52.571486+0000","last_peered":"2024-09-22T03:28:52.571486+0000","last_clean":"2024-09-22T03:28:52.571486+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:28:52.571486+0000","last_undegraded":"2024-09-22T03:28:52.571486+0000","last_fullsized":"2024-09-22T03:28:52.571486+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":601,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.174178+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:28:51.174178+0000","last_peered":"2024-09-22T03:28:51.174178+0000","last_clean":"2024-09-22T03:28:51.174178+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:28:51.174178+0000","last_undegraded":"2024-09-22T03:28:51.174178+0000","last_fullsized":"2024-09-22T03:28:51.174178+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"216'2229","reported_seq":2558,"reported_epoch":216,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:51.871115+0000","last_change":"2024-09-22T03:28:49.464952+0000","last_active":"2024-09-22T03:28:51.871115+0000","last_peered":"2024-09-22T03:28:51.871115+0000","last_clean":"2024-09-22T03:28:07.415668+0000","last_became_active":"2024-09-22T03:28:43.454069+0000","last_became_peered":"2024-09-22T03:28:43.454069+0000","last_unstale":"2024-09-22T03:28:51.871115+0000","last_undegraded":"2024-09-22T03:28:51.871115+0000","last_fullsized":"2024-09-22T03:28:43.296775+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":43}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":318,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.169643+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:28:51.169643+0000","last_peered":"2024-09-22T03:28:51.169643+0000","last_clean":"2024-09-22T03:28:51.169643+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:28:51.169643+0000","last_undegraded":"2024-09-22T03:28:51.169643+0000","last_fullsized":"2024-09-22T03:28:51.169643+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"216'2186","reported_seq":2866,"reported_epoch":216,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:51.682362+0000","last_change":"2024-09-22T03:28:43.548959+0000","last_active":"2024-09-22T03:28:51.682362+0000","last_peered":"2024-09-22T03:28:51.682362+0000","last_clean":"2024-09-22T03:28:07.289540+0000","last_became_active":"2024-09-22T03:28:43.542933+0000","last_became_peered":"2024-09-22T03:28:43.542933+0000","last_unstale":"2024-09-22T03:28:51.682362+0000","last_undegraded":"2024-09-22T03:28:43.262227+0000","last_fullsized":"2024-09-22T03:28:43.231436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2187,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":445,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.542086+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:28:52.542086+0000","last_peered":"2024-09-22T03:28:52.542086+0000","last_clean":"2024-09-22T03:28:52.542086+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:28:52.542086+0000","last_undegraded":"2024-09-22T03:28:52.542086+0000","last_fullsized":"2024-09-22T03:28:52.542086+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"217'2302","reported_seq":2996,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:52.964432+0000","last_change":"2024-09-22T03:28:43.639120+0000","last_active":"2024-09-22T03:28:52.964432+0000","last_peered":"2024-09-22T03:28:52.964432+0000","last_clean":"2024-09-22T03:28:07.504961+0000","last_became_active":"2024-09-22T03:28:43.635629+0000","last_became_peered":"2024-09-22T03:28:43.635629+0000","last_unstale":"2024-09-22T03:28:52.964432+0000","last_undegraded":"2024-09-22T03:28:43.522617+0000","last_fullsized":"2024-09-22T03:28:43.457119+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2303,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":389,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.169956+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:28:51.169956+0000","last_peered":"2024-09-22T03:28:51.169956+0000","last_clean":"2024-09-22T03:28:51.169956+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:28:51.169956+0000","last_undegraded":"2024-09-22T03:28:51.169956+0000","last_fullsized":"2024-09-22T03:28:51.169956+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"217'2273","reported_seq":2872,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.835313+0000","last_change":"2024-09-22T03:27:57.341822+0000","last_active":"2024-09-22T03:28:52.835313+0000","last_peered":"2024-09-22T03:28:52.835313+0000","last_clean":"2024-09-22T03:28:52.835313+0000","last_became_active":"2024-09-22T03:27:57.326249+0000","last_became_peered":"2024-09-22T03:27:57.326249+0000","last_unstale":"2024-09-22T03:28:52.835313+0000","last_undegraded":"2024-09-22T03:28:52.835313+0000","last_fullsized":"2024-09-22T03:28:52.835313+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":395,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.168386+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:28:51.168386+0000","last_peered":"2024-09-22T03:28:51.168386+0000","last_clean":"2024-09-22T03:28:51.168386+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:28:51.168386+0000","last_undegraded":"2024-09-22T03:28:51.168386+0000","last_fullsized":"2024-09-22T03:28:51.168386+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":32112664,"num_objects":491,"num_object_clones":0,"num_object_copies":1473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":35557,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3724,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12627968,"data_stored":201850952,"data_compressed":9584960,"data_compressed_allocated":12615680,"data_compressed_original":201850880,"omap_allocated":0,"internal_metadata":0},"log_size":35545,"ondisk_log_size":35545,"up":48,"acting":38,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599302,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101896,"kb_used_data":3064,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":94269944,"statfs":{"total":96636764160,"available":96532422656,"internally_reserved":0,"allocated":3137536,"data_stored":22239159,"data_compressed":1020736,"data_compressed_allocated":1343488,"data_compressed_original":21495808,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41020,"kb_used_data":3244,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94330820,"statfs":{"total":96636764160,"available":96594759680,"internally_reserved":0,"allocated":3321856,"data_stored":24861119,"data_compressed":1147180,"data_compressed_allocated":1511424,"data_compressed_original":24100864,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52200,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":42112,"kb_avail":94319640,"statfs":{"total":96636764160,"available":96583311360,"internally_reserved":0,"allocated":10313728,"data_stored":136665559,"data_compressed":6456252,"data_compressed_allocated":8499200,"data_compressed_original":135905280,"omap_allocated":0,"internal_metadata":43122688},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328439,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46888,"kb_used_data":4568,"kb_used_omap":0,"kb_used_meta":42304,"kb_avail":94324952,"statfs":{"total":96636764160,"available":96588750848,"internally_reserved":0,"allocated":4677632,"data_stored":46553540,"data_compressed":2177252,"data_compressed_allocated":2867200,"data_compressed_original":45793280,"omap_allocated":0,"internal_metadata":43319296},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2265088,"data_stored":36175896,"data_compressed":1717824,"data_compressed_allocated":2260992,"data_compressed_original":36175872,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8343552,"data_stored":133431320,"data_compressed":6336032,"data_compressed_allocated":8339456,"data_compressed_original":133431296,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1052672,"data_stored":16842752,"data_compressed":799784,"data_compressed_allocated":1052672,"data_compressed_original":16842752,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":15400984,"data_compressed":731320,"data_compressed_allocated":962560,"data_compressed_original":15400960,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:54.040 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:54.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:54.180+0000 7fd879b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:54.191+0000 7fd879b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.511 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:54.512 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:28:54.528 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":746,"stamp":"2024-09-22T03:28:53.533060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32703051,"num_objects":495,"num_object_clones":0,"num_object_copies":1485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":35696,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3724,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35645,"ondisk_log_size":35645,"up":75,"acting":65,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":242004,"kb_used_data":20948,"kb_used_omap":0,"kb_used_meta":220992,"kb_avail":377245356,"statfs":{"total":386547056640,"available":386299244544,"internally_reserved":0,"allocated":21450752,"data_stored":230319377,"data_compressed":10801420,"data_compressed_allocated":14221312,"data_compressed_original":227295232,"omap_allocated":0,"internal_metadata":226295808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-59965440,"num_objects":-915,"num_object_clones":0,"num_object_copies":-2745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":915,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010047"},"pg_stats":[{"pgid":"6.f","version":"217'2234","reported_seq":3180,"reported_epoch":217,"state":"active+recovery_wait","last_fresh":"2024-09-22T03:28:52.828389+0000","last_change":"2024-09-22T03:28:49.630613+0000","last_active":"2024-09-22T03:28:52.828389+0000","last_peered":"2024-09-22T03:28:52.828389+0000","last_clean":"2024-09-22T03:28:07.444665+0000","last_became_active":"2024-09-22T03:28:49.595919+0000","last_became_peered":"2024-09-22T03:28:49.595919+0000","last_unstale":"2024-09-22T03:28:52.828389+0000","last_undegraded":"2024-09-22T03:28:52.828389+0000","last_fullsized":"2024-09-22T03:28:52.828389+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2235,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"0,1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"216'2210","reported_seq":3188,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.321730+0000","last_change":"2024-09-22T03:28:51.207954+0000","last_active":"2024-09-22T03:28:51.321730+0000","last_peered":"2024-09-22T03:28:51.321730+0000","last_clean":"2024-09-22T03:28:51.321730+0000","last_became_active":"2024-09-22T03:28:51.187017+0000","last_became_peered":"2024-09-22T03:28:51.187017+0000","last_unstale":"2024-09-22T03:28:51.321730+0000","last_undegraded":"2024-09-22T03:28:51.321730+0000","last_fullsized":"2024-09-22T03:28:51.321730+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2210,"log_dups_size":0,"ondisk_log_size":2210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2211,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"216'2138","reported_seq":2614,"reported_epoch":216,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:51.887493+0000","last_change":"2024-09-22T03:28:48.040111+0000","last_active":"2024-09-22T03:28:51.887493+0000","last_peered":"2024-09-22T03:28:51.887493+0000","last_clean":"2024-09-22T03:28:07.418415+0000","last_became_active":"2024-09-22T03:28:43.449859+0000","last_became_peered":"2024-09-22T03:28:43.449859+0000","last_unstale":"2024-09-22T03:28:51.887493+0000","last_undegraded":"2024-09-22T03:28:51.887493+0000","last_fullsized":"2024-09-22T03:28:43.266518+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2687000,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2138,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.c","version":"217'2313","reported_seq":3014,"reported_epoch":217,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:53.150846+0000","last_change":"2024-09-22T03:28:52.285232+0000","last_active":"2024-09-22T03:28:53.150846+0000","last_peered":"2024-09-22T03:28:53.150846+0000","last_clean":"2024-09-22T03:28:07.483127+0000","last_became_active":"2024-09-22T03:28:43.555328+0000","last_became_peered":"2024-09-22T03:28:43.555328+0000","last_unstale":"2024-09-22T03:28:53.150846+0000","last_undegraded":"2024-09-22T03:28:53.150846+0000","last_fullsized":"2024-09-22T03:28:43.259727+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2313,"log_dups_size":0,"ondisk_log_size":2313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":17}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"217'2258","reported_seq":2899,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:52.966321+0000","last_change":"2024-09-22T03:28:43.568261+0000","last_active":"2024-09-22T03:28:52.966321+0000","last_peered":"2024-09-22T03:28:52.966321+0000","last_clean":"2024-09-22T03:28:07.431554+0000","last_became_active":"2024-09-22T03:28:43.562050+0000","last_became_peered":"2024-09-22T03:28:43.562050+0000","last_unstale":"2024-09-22T03:28:52.966321+0000","last_undegraded":"2024-09-22T03:28:43.414054+0000","last_fullsized":"2024-09-22T03:28:43.387740+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2259,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"217'2186","reported_seq":3134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.805946+0000","last_change":"2024-09-22T03:28:50.193603+0000","last_active":"2024-09-22T03:28:52.805946+0000","last_peered":"2024-09-22T03:28:52.805946+0000","last_clean":"2024-09-22T03:28:52.805946+0000","last_became_active":"2024-09-22T03:28:50.182409+0000","last_became_peered":"2024-09-22T03:28:50.182409+0000","last_unstale":"2024-09-22T03:28:52.805946+0000","last_undegraded":"2024-09-22T03:28:52.805946+0000","last_fullsized":"2024-09-22T03:28:52.805946+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2187,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"217'2179","reported_seq":3042,"reported_epoch":217,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:53.142528+0000","last_change":"2024-09-22T03:28:50.020409+0000","last_active":"2024-09-22T03:28:53.142528+0000","last_peered":"2024-09-22T03:28:53.142528+0000","last_clean":"2024-09-22T03:28:07.457799+0000","last_became_active":"2024-09-22T03:28:43.547696+0000","last_became_peered":"2024-09-22T03:28:43.547696+0000","last_unstale":"2024-09-22T03:28:53.142528+0000","last_undegraded":"2024-09-22T03:28:53.142528+0000","last_fullsized":"2024-09-22T03:28:43.229591+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2179,"log_dups_size":0,"ondisk_log_size":2179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":15}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"216'2100","reported_seq":2691,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.661009+0000","last_change":"2024-09-22T03:27:57.323000+0000","last_active":"2024-09-22T03:28:51.661009+0000","last_peered":"2024-09-22T03:28:51.661009+0000","last_clean":"2024-09-22T03:28:51.661009+0000","last_became_active":"2024-09-22T03:27:57.315204+0000","last_became_peered":"2024-09-22T03:27:57.315204+0000","last_unstale":"2024-09-22T03:28:51.661009+0000","last_undegraded":"2024-09-22T03:28:51.661009+0000","last_fullsized":"2024-09-22T03:28:51.661009+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2100,"log_dups_size":0,"ondisk_log_size":2100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2101,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"216'2125","reported_seq":2310,"reported_epoch":216,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:51.698393+0000","last_change":"2024-09-22T03:28:43.750347+0000","last_active":"2024-09-22T03:28:51.698393+0000","last_peered":"2024-09-22T03:28:51.698393+0000","last_clean":"2024-09-22T03:28:07.505453+0000","last_became_active":"2024-09-22T03:28:43.459458+0000","last_became_peered":"2024-09-22T03:28:43.459458+0000","last_unstale":"2024-09-22T03:28:51.698393+0000","last_undegraded":"2024-09-22T03:28:43.744545+0000","last_fullsized":"2024-09-22T03:28:43.299387+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2125,"log_dups_size":0,"ondisk_log_size":2125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2126,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":426,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.571925+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:28:52.571925+0000","last_peered":"2024-09-22T03:28:52.571925+0000","last_clean":"2024-09-22T03:28:52.571925+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:28:52.571925+0000","last_undegraded":"2024-09-22T03:28:52.571925+0000","last_fullsized":"2024-09-22T03:28:52.571925+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"217'2305","reported_seq":2528,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:53.048381+0000","last_change":"2024-09-22T03:28:43.663585+0000","last_active":"2024-09-22T03:28:53.048381+0000","last_peered":"2024-09-22T03:28:53.048381+0000","last_clean":"2024-09-22T03:28:07.512449+0000","last_became_active":"2024-09-22T03:28:43.415468+0000","last_became_peered":"2024-09-22T03:28:43.415468+0000","last_unstale":"2024-09-22T03:28:53.048381+0000","last_undegraded":"2024-09-22T03:28:43.660936+0000","last_fullsized":"2024-09-22T03:28:43.261374+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":484,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.168452+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:28:51.168452+0000","last_peered":"2024-09-22T03:28:51.168452+0000","last_clean":"2024-09-22T03:28:51.168452+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:28:51.168452+0000","last_undegraded":"2024-09-22T03:28:51.168452+0000","last_fullsized":"2024-09-22T03:28:51.168452+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"216'2244","reported_seq":2776,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.540578+0000","last_change":"2024-09-22T03:27:57.489568+0000","last_active":"2024-09-22T03:28:52.540578+0000","last_peered":"2024-09-22T03:28:52.540578+0000","last_clean":"2024-09-22T03:28:52.540578+0000","last_became_active":"2024-09-22T03:27:57.314117+0000","last_became_peered":"2024-09-22T03:27:57.314117+0000","last_unstale":"2024-09-22T03:28:52.540578+0000","last_undegraded":"2024-09-22T03:28:52.540578+0000","last_fullsized":"2024-09-22T03:28:52.540578+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2244,"log_dups_size":0,"ondisk_log_size":2244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":551,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.544931+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:28:52.544931+0000","last_peered":"2024-09-22T03:28:52.544931+0000","last_clean":"2024-09-22T03:28:52.544931+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:28:52.544931+0000","last_undegraded":"2024-09-22T03:28:52.544931+0000","last_fullsized":"2024-09-22T03:28:52.544931+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"217'2263","reported_seq":2516,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:53.152011+0000","last_change":"2024-09-22T03:28:43.598307+0000","last_active":"2024-09-22T03:28:53.152011+0000","last_peered":"2024-09-22T03:28:53.152011+0000","last_clean":"2024-09-22T03:28:07.488866+0000","last_became_active":"2024-09-22T03:28:43.420961+0000","last_became_peered":"2024-09-22T03:28:43.420961+0000","last_unstale":"2024-09-22T03:28:53.152011+0000","last_undegraded":"2024-09-22T03:28:43.596998+0000","last_fullsized":"2024-09-22T03:28:43.260012+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2264,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":517,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.571486+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:28:52.571486+0000","last_peered":"2024-09-22T03:28:52.571486+0000","last_clean":"2024-09-22T03:28:52.571486+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:28:52.571486+0000","last_undegraded":"2024-09-22T03:28:52.571486+0000","last_fullsized":"2024-09-22T03:28:52.571486+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":601,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.174178+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:28:51.174178+0000","last_peered":"2024-09-22T03:28:51.174178+0000","last_clean":"2024-09-22T03:28:51.174178+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:28:51.174178+0000","last_undegraded":"2024-09-22T03:28:51.174178+0000","last_fullsized":"2024-09-22T03:28:51.174178+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"216'2229","reported_seq":2558,"reported_epoch":216,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:51.871115+0000","last_change":"2024-09-22T03:28:49.464952+0000","last_active":"2024-09-22T03:28:51.871115+0000","last_peered":"2024-09-22T03:28:51.871115+0000","last_clean":"2024-09-22T03:28:07.415668+0000","last_became_active":"2024-09-22T03:28:43.454069+0000","last_became_peered":"2024-09-22T03:28:43.454069+0000","last_unstale":"2024-09-22T03:28:51.871115+0000","last_undegraded":"2024-09-22T03:28:51.871115+0000","last_fullsized":"2024-09-22T03:28:43.296775+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":43}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":318,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.169643+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:28:51.169643+0000","last_peered":"2024-09-22T03:28:51.169643+0000","last_clean":"2024-09-22T03:28:51.169643+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:28:51.169643+0000","last_undegraded":"2024-09-22T03:28:51.169643+0000","last_fullsized":"2024-09-22T03:28:51.169643+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"216'2186","reported_seq":2866,"reported_epoch":216,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:51.682362+0000","last_change":"2024-09-22T03:28:43.548959+0000","last_active":"2024-09-22T03:28:51.682362+0000","last_peered":"2024-09-22T03:28:51.682362+0000","last_clean":"2024-09-22T03:28:07.289540+0000","last_became_active":"2024-09-22T03:28:43.542933+0000","last_became_peered":"2024-09-22T03:28:43.542933+0000","last_unstale":"2024-09-22T03:28:51.682362+0000","last_undegraded":"2024-09-22T03:28:43.262227+0000","last_fullsized":"2024-09-22T03:28:43.231436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2187,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":445,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.542086+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:28:52.542086+0000","last_peered":"2024-09-22T03:28:52.542086+0000","last_clean":"2024-09-22T03:28:52.542086+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:28:52.542086+0000","last_undegraded":"2024-09-22T03:28:52.542086+0000","last_fullsized":"2024-09-22T03:28:52.542086+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"217'2302","reported_seq":2996,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:52.964432+0000","last_change":"2024-09-22T03:28:43.639120+0000","last_active":"2024-09-22T03:28:52.964432+0000","last_peered":"2024-09-22T03:28:52.964432+0000","last_clean":"2024-09-22T03:28:07.504961+0000","last_became_active":"2024-09-22T03:28:43.635629+0000","last_became_peered":"2024-09-22T03:28:43.635629+0000","last_unstale":"2024-09-22T03:28:52.964432+0000","last_undegraded":"2024-09-22T03:28:43.522617+0000","last_fullsized":"2024-09-22T03:28:43.457119+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2303,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":389,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.169956+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:28:51.169956+0000","last_peered":"2024-09-22T03:28:51.169956+0000","last_clean":"2024-09-22T03:28:51.169956+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:28:51.169956+0000","last_undegraded":"2024-09-22T03:28:51.169956+0000","last_fullsized":"2024-09-22T03:28:51.169956+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"217'2273","reported_seq":2872,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.835313+0000","last_change":"2024-09-22T03:27:57.341822+0000","last_active":"2024-09-22T03:28:52.835313+0000","last_peered":"2024-09-22T03:28:52.835313+0000","last_clean":"2024-09-22T03:28:52.835313+0000","last_became_active":"2024-09-22T03:27:57.326249+0000","last_became_peered":"2024-09-22T03:27:57.326249+0000","last_unstale":"2024-09-22T03:28:52.835313+0000","last_undegraded":"2024-09-22T03:28:52.835313+0000","last_fullsized":"2024-09-22T03:28:52.835313+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":395,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.168386+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:28:51.168386+0000","last_peered":"2024-09-22T03:28:51.168386+0000","last_clean":"2024-09-22T03:28:51.168386+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:28:51.168386+0000","last_undegraded":"2024-09-22T03:28:51.168386+0000","last_fullsized":"2024-09-22T03:28:51.168386+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":32112664,"num_objects":491,"num_object_clones":0,"num_object_copies":1473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":35557,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3724,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12627968,"data_stored":201850952,"data_compressed":9584960,"data_compressed_allocated":12615680,"data_compressed_original":201850880,"omap_allocated":0,"internal_metadata":0},"log_size":35545,"ondisk_log_size":35545,"up":48,"acting":38,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599302,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101896,"kb_used_data":3064,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":94269944,"statfs":{"total":96636764160,"available":96532422656,"internally_reserved":0,"allocated":3137536,"data_stored":22239159,"data_compressed":1020736,"data_compressed_allocated":1343488,"data_compressed_original":21495808,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41020,"kb_used_data":3244,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94330820,"statfs":{"total":96636764160,"available":96594759680,"internally_reserved":0,"allocated":3321856,"data_stored":24861119,"data_compressed":1147180,"data_compressed_allocated":1511424,"data_compressed_original":24100864,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52200,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":42112,"kb_avail":94319640,"statfs":{"total":96636764160,"available":96583311360,"internally_reserved":0,"allocated":10313728,"data_stored":136665559,"data_compressed":6456252,"data_compressed_allocated":8499200,"data_compressed_original":135905280,"omap_allocated":0,"internal_metadata":43122688},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328439,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46888,"kb_used_data":4568,"kb_used_omap":0,"kb_used_meta":42304,"kb_avail":94324952,"statfs":{"total":96636764160,"available":96588750848,"internally_reserved":0,"allocated":4677632,"data_stored":46553540,"data_compressed":2177252,"data_compressed_allocated":2867200,"data_compressed_original":45793280,"omap_allocated":0,"internal_metadata":43319296},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2265088,"data_stored":36175896,"data_compressed":1717824,"data_compressed_allocated":2260992,"data_compressed_original":36175872,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8343552,"data_stored":133431320,"data_compressed":6336032,"data_compressed_allocated":8339456,"data_compressed_original":133431296,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1052672,"data_stored":16842752,"data_compressed":799784,"data_compressed_allocated":1052672,"data_compressed_original":16842752,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":15400984,"data_compressed":731320,"data_compressed_allocated":962560,"data_compressed_original":15400960,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:54.530 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:28:54.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:54.682+0000 7ff2d8c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:54.692+0000 7ff2d8c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.745 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Removed 18023 objects 2024-09-22T03:28:54.745 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Clean up completed and total clean up time :157.178 2024-09-22T03:28:54.995 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:54.995 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 964/1485 objects degraded (64.916%), 6 pgs degraded","count":6},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":1079,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":218,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975721,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+remapped","count":4},{"state_name":"active+recovery_wait","count":1}],"num_pgs":25,"num_pools":3,"num_objects":495,"data_bytes":32703051,"bytes_used":247812096,"bytes_avail":386299244544,"bytes_total":386547056640,"degraded_objects":964,"degraded_total":1485,"degraded_ratio":0.64915824915824916,"recovering_objects_per_sec":127,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":509,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":228},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T03:27:01.429211+0000","services":{}},"progress_events":{}} 2024-09-22T03:28:55.009 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T03:28:55.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:55.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:55.124+0000 7f8380c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:55.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:55.133+0000 7f8380c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:55.338 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:55.339 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:28:55.353 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":747,"stamp":"2024-09-22T03:28:55.153097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32703051,"num_objects":495,"num_object_clones":0,"num_object_copies":1485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":35696,"num_write_kb":1155369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3724,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35645,"ondisk_log_size":35645,"up":75,"acting":65,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":242004,"kb_used_data":20948,"kb_used_omap":0,"kb_used_meta":220992,"kb_avail":377245356,"statfs":{"total":386547056640,"available":386299244544,"internally_reserved":0,"allocated":21450752,"data_stored":230319377,"data_compressed":10801420,"data_compressed_allocated":14221312,"data_compressed_original":227295232,"omap_allocated":0,"internal_metadata":226295808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-59965440,"num_objects":-915,"num_object_clones":0,"num_object_copies":-2745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":915,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.003953"},"pg_stats":[{"pgid":"6.f","version":"217'2234","reported_seq":3180,"reported_epoch":217,"state":"active+recovery_wait","last_fresh":"2024-09-22T03:28:52.828389+0000","last_change":"2024-09-22T03:28:49.630613+0000","last_active":"2024-09-22T03:28:52.828389+0000","last_peered":"2024-09-22T03:28:52.828389+0000","last_clean":"2024-09-22T03:28:07.444665+0000","last_became_active":"2024-09-22T03:28:49.595919+0000","last_became_peered":"2024-09-22T03:28:49.595919+0000","last_unstale":"2024-09-22T03:28:52.828389+0000","last_undegraded":"2024-09-22T03:28:52.828389+0000","last_fullsized":"2024-09-22T03:28:52.828389+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2235,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"0,1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"216'2210","reported_seq":3188,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.321730+0000","last_change":"2024-09-22T03:28:51.207954+0000","last_active":"2024-09-22T03:28:51.321730+0000","last_peered":"2024-09-22T03:28:51.321730+0000","last_clean":"2024-09-22T03:28:51.321730+0000","last_became_active":"2024-09-22T03:28:51.187017+0000","last_became_peered":"2024-09-22T03:28:51.187017+0000","last_unstale":"2024-09-22T03:28:51.321730+0000","last_undegraded":"2024-09-22T03:28:51.321730+0000","last_fullsized":"2024-09-22T03:28:51.321730+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2210,"log_dups_size":0,"ondisk_log_size":2210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2211,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"216'2138","reported_seq":2614,"reported_epoch":216,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:51.887493+0000","last_change":"2024-09-22T03:28:48.040111+0000","last_active":"2024-09-22T03:28:51.887493+0000","last_peered":"2024-09-22T03:28:51.887493+0000","last_clean":"2024-09-22T03:28:07.418415+0000","last_became_active":"2024-09-22T03:28:43.449859+0000","last_became_peered":"2024-09-22T03:28:43.449859+0000","last_unstale":"2024-09-22T03:28:51.887493+0000","last_undegraded":"2024-09-22T03:28:51.887493+0000","last_fullsized":"2024-09-22T03:28:43.266518+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2687000,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2138,"num_write_kb":69697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.c","version":"217'2313","reported_seq":3014,"reported_epoch":217,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:53.150846+0000","last_change":"2024-09-22T03:28:52.285232+0000","last_active":"2024-09-22T03:28:53.150846+0000","last_peered":"2024-09-22T03:28:53.150846+0000","last_clean":"2024-09-22T03:28:07.483127+0000","last_became_active":"2024-09-22T03:28:43.555328+0000","last_became_peered":"2024-09-22T03:28:43.555328+0000","last_unstale":"2024-09-22T03:28:53.150846+0000","last_undegraded":"2024-09-22T03:28:53.150846+0000","last_fullsized":"2024-09-22T03:28:43.259727+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2313,"log_dups_size":0,"ondisk_log_size":2313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":74560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":17}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"217'2258","reported_seq":2899,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:52.966321+0000","last_change":"2024-09-22T03:28:43.568261+0000","last_active":"2024-09-22T03:28:52.966321+0000","last_peered":"2024-09-22T03:28:52.966321+0000","last_clean":"2024-09-22T03:28:07.431554+0000","last_became_active":"2024-09-22T03:28:43.562050+0000","last_became_peered":"2024-09-22T03:28:43.562050+0000","last_unstale":"2024-09-22T03:28:52.966321+0000","last_undegraded":"2024-09-22T03:28:43.414054+0000","last_fullsized":"2024-09-22T03:28:43.387740+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2259,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"217'2186","reported_seq":3134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.805946+0000","last_change":"2024-09-22T03:28:50.193603+0000","last_active":"2024-09-22T03:28:52.805946+0000","last_peered":"2024-09-22T03:28:52.805946+0000","last_clean":"2024-09-22T03:28:52.805946+0000","last_became_active":"2024-09-22T03:28:50.182409+0000","last_became_peered":"2024-09-22T03:28:50.182409+0000","last_unstale":"2024-09-22T03:28:52.805946+0000","last_undegraded":"2024-09-22T03:28:52.805946+0000","last_fullsized":"2024-09-22T03:28:52.805946+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2187,"num_write_kb":70784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"217'2179","reported_seq":3042,"reported_epoch":217,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:53.142528+0000","last_change":"2024-09-22T03:28:50.020409+0000","last_active":"2024-09-22T03:28:53.142528+0000","last_peered":"2024-09-22T03:28:53.142528+0000","last_clean":"2024-09-22T03:28:07.457799+0000","last_became_active":"2024-09-22T03:28:43.547696+0000","last_became_peered":"2024-09-22T03:28:43.547696+0000","last_unstale":"2024-09-22T03:28:53.142528+0000","last_undegraded":"2024-09-22T03:28:53.142528+0000","last_fullsized":"2024-09-22T03:28:43.229591+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2179,"log_dups_size":0,"ondisk_log_size":2179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":70208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":15}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"216'2100","reported_seq":2691,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.661009+0000","last_change":"2024-09-22T03:27:57.323000+0000","last_active":"2024-09-22T03:28:51.661009+0000","last_peered":"2024-09-22T03:28:51.661009+0000","last_clean":"2024-09-22T03:28:51.661009+0000","last_became_active":"2024-09-22T03:27:57.315204+0000","last_became_peered":"2024-09-22T03:27:57.315204+0000","last_unstale":"2024-09-22T03:28:51.661009+0000","last_undegraded":"2024-09-22T03:28:51.661009+0000","last_fullsized":"2024-09-22T03:28:51.661009+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2100,"log_dups_size":0,"ondisk_log_size":2100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2101,"num_write_kb":68608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"216'2125","reported_seq":2310,"reported_epoch":216,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:51.698393+0000","last_change":"2024-09-22T03:28:43.750347+0000","last_active":"2024-09-22T03:28:51.698393+0000","last_peered":"2024-09-22T03:28:51.698393+0000","last_clean":"2024-09-22T03:28:07.505453+0000","last_became_active":"2024-09-22T03:28:43.459458+0000","last_became_peered":"2024-09-22T03:28:43.459458+0000","last_unstale":"2024-09-22T03:28:51.698393+0000","last_undegraded":"2024-09-22T03:28:43.744545+0000","last_fullsized":"2024-09-22T03:28:43.299387+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2125,"log_dups_size":0,"ondisk_log_size":2125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2126,"num_write_kb":69312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":426,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.571925+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:28:52.571925+0000","last_peered":"2024-09-22T03:28:52.571925+0000","last_clean":"2024-09-22T03:28:52.571925+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:28:52.571925+0000","last_undegraded":"2024-09-22T03:28:52.571925+0000","last_fullsized":"2024-09-22T03:28:52.571925+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"217'2305","reported_seq":2528,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:53.048381+0000","last_change":"2024-09-22T03:28:43.663585+0000","last_active":"2024-09-22T03:28:53.048381+0000","last_peered":"2024-09-22T03:28:53.048381+0000","last_clean":"2024-09-22T03:28:07.512449+0000","last_became_active":"2024-09-22T03:28:43.415468+0000","last_became_peered":"2024-09-22T03:28:43.415468+0000","last_unstale":"2024-09-22T03:28:53.048381+0000","last_undegraded":"2024-09-22T03:28:43.660936+0000","last_fullsized":"2024-09-22T03:28:43.261374+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":74368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":484,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.168452+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:28:51.168452+0000","last_peered":"2024-09-22T03:28:51.168452+0000","last_clean":"2024-09-22T03:28:51.168452+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:28:51.168452+0000","last_undegraded":"2024-09-22T03:28:51.168452+0000","last_fullsized":"2024-09-22T03:28:51.168452+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"216'2244","reported_seq":2776,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.540578+0000","last_change":"2024-09-22T03:27:57.489568+0000","last_active":"2024-09-22T03:28:52.540578+0000","last_peered":"2024-09-22T03:28:52.540578+0000","last_clean":"2024-09-22T03:28:52.540578+0000","last_became_active":"2024-09-22T03:27:57.314117+0000","last_became_peered":"2024-09-22T03:27:57.314117+0000","last_unstale":"2024-09-22T03:28:52.540578+0000","last_undegraded":"2024-09-22T03:28:52.540578+0000","last_fullsized":"2024-09-22T03:28:52.540578+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2244,"log_dups_size":0,"ondisk_log_size":2244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":72640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":551,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.544931+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:28:52.544931+0000","last_peered":"2024-09-22T03:28:52.544931+0000","last_clean":"2024-09-22T03:28:52.544931+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:28:52.544931+0000","last_undegraded":"2024-09-22T03:28:52.544931+0000","last_fullsized":"2024-09-22T03:28:52.544931+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"217'2263","reported_seq":2516,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:53.152011+0000","last_change":"2024-09-22T03:28:43.598307+0000","last_active":"2024-09-22T03:28:53.152011+0000","last_peered":"2024-09-22T03:28:53.152011+0000","last_clean":"2024-09-22T03:28:07.488866+0000","last_became_active":"2024-09-22T03:28:43.420961+0000","last_became_peered":"2024-09-22T03:28:43.420961+0000","last_unstale":"2024-09-22T03:28:53.152011+0000","last_undegraded":"2024-09-22T03:28:43.596998+0000","last_fullsized":"2024-09-22T03:28:43.260012+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2264,"num_write_kb":73152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":517,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.571486+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:28:52.571486+0000","last_peered":"2024-09-22T03:28:52.571486+0000","last_clean":"2024-09-22T03:28:52.571486+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:28:52.571486+0000","last_undegraded":"2024-09-22T03:28:52.571486+0000","last_fullsized":"2024-09-22T03:28:52.571486+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":601,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.174178+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:28:51.174178+0000","last_peered":"2024-09-22T03:28:51.174178+0000","last_clean":"2024-09-22T03:28:51.174178+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:28:51.174178+0000","last_undegraded":"2024-09-22T03:28:51.174178+0000","last_fullsized":"2024-09-22T03:28:51.174178+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"216'2229","reported_seq":2558,"reported_epoch":216,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:28:51.871115+0000","last_change":"2024-09-22T03:28:49.464952+0000","last_active":"2024-09-22T03:28:51.871115+0000","last_peered":"2024-09-22T03:28:51.871115+0000","last_clean":"2024-09-22T03:28:07.415668+0000","last_became_active":"2024-09-22T03:28:43.454069+0000","last_became_peered":"2024-09-22T03:28:43.454069+0000","last_unstale":"2024-09-22T03:28:51.871115+0000","last_undegraded":"2024-09-22T03:28:51.871115+0000","last_fullsized":"2024-09-22T03:28:43.296775+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":72704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":43}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":318,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.169643+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:28:51.169643+0000","last_peered":"2024-09-22T03:28:51.169643+0000","last_clean":"2024-09-22T03:28:51.169643+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:28:51.169643+0000","last_undegraded":"2024-09-22T03:28:51.169643+0000","last_fullsized":"2024-09-22T03:28:51.169643+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"216'2186","reported_seq":2866,"reported_epoch":216,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:51.682362+0000","last_change":"2024-09-22T03:28:43.548959+0000","last_active":"2024-09-22T03:28:51.682362+0000","last_peered":"2024-09-22T03:28:51.682362+0000","last_clean":"2024-09-22T03:28:07.289540+0000","last_became_active":"2024-09-22T03:28:43.542933+0000","last_became_peered":"2024-09-22T03:28:43.542933+0000","last_unstale":"2024-09-22T03:28:51.682362+0000","last_undegraded":"2024-09-22T03:28:43.262227+0000","last_fullsized":"2024-09-22T03:28:43.231436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2187,"num_write_kb":71424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":445,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.542086+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:28:52.542086+0000","last_peered":"2024-09-22T03:28:52.542086+0000","last_clean":"2024-09-22T03:28:52.542086+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:28:52.542086+0000","last_undegraded":"2024-09-22T03:28:52.542086+0000","last_fullsized":"2024-09-22T03:28:52.542086+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"217'2302","reported_seq":2996,"reported_epoch":217,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:28:52.964432+0000","last_change":"2024-09-22T03:28:43.639120+0000","last_active":"2024-09-22T03:28:52.964432+0000","last_peered":"2024-09-22T03:28:52.964432+0000","last_clean":"2024-09-22T03:28:07.504961+0000","last_became_active":"2024-09-22T03:28:43.635629+0000","last_became_peered":"2024-09-22T03:28:43.635629+0000","last_unstale":"2024-09-22T03:28:52.964432+0000","last_undegraded":"2024-09-22T03:28:43.522617+0000","last_fullsized":"2024-09-22T03:28:43.457119+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2303,"num_write_kb":74496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":389,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.169956+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:28:51.169956+0000","last_peered":"2024-09-22T03:28:51.169956+0000","last_clean":"2024-09-22T03:28:51.169956+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:28:51.169956+0000","last_undegraded":"2024-09-22T03:28:51.169956+0000","last_fullsized":"2024-09-22T03:28:51.169956+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"217'2273","reported_seq":2872,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-22T03:28:52.835313+0000","last_change":"2024-09-22T03:27:57.341822+0000","last_active":"2024-09-22T03:28:52.835313+0000","last_peered":"2024-09-22T03:28:52.835313+0000","last_clean":"2024-09-22T03:28:52.835313+0000","last_became_active":"2024-09-22T03:27:57.326249+0000","last_became_peered":"2024-09-22T03:27:57.326249+0000","last_unstale":"2024-09-22T03:28:52.835313+0000","last_undegraded":"2024-09-22T03:28:52.835313+0000","last_fullsized":"2024-09-22T03:28:52.835313+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:24:19.449317+0000","last_clean_scrub_stamp":"2024-09-22T03:24:19.449317+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":73408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":395,"reported_epoch":216,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.168386+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:28:51.168386+0000","last_peered":"2024-09-22T03:28:51.168386+0000","last_clean":"2024-09-22T03:28:51.168386+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:28:51.168386+0000","last_undegraded":"2024-09-22T03:28:51.168386+0000","last_fullsized":"2024-09-22T03:28:51.168386+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":32112664,"num_objects":491,"num_object_clones":0,"num_object_copies":1473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":35557,"num_write_kb":1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3724,"num_bytes_recovered":1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12627968,"data_stored":201850952,"data_compressed":9584960,"data_compressed_allocated":12615680,"data_compressed_original":201850880,"omap_allocated":0,"internal_metadata":0},"log_size":35545,"ondisk_log_size":35545,"up":48,"acting":38,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599302,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101896,"kb_used_data":3064,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":94269944,"statfs":{"total":96636764160,"available":96532422656,"internally_reserved":0,"allocated":3137536,"data_stored":22239159,"data_compressed":1020736,"data_compressed_allocated":1343488,"data_compressed_original":21495808,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41020,"kb_used_data":3244,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94330820,"statfs":{"total":96636764160,"available":96594759680,"internally_reserved":0,"allocated":3321856,"data_stored":24861119,"data_compressed":1147180,"data_compressed_allocated":1511424,"data_compressed_original":24100864,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52200,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":42112,"kb_avail":94319640,"statfs":{"total":96636764160,"available":96583311360,"internally_reserved":0,"allocated":10313728,"data_stored":136665559,"data_compressed":6456252,"data_compressed_allocated":8499200,"data_compressed_original":135905280,"omap_allocated":0,"internal_metadata":43122688},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328439,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46888,"kb_used_data":4568,"kb_used_omap":0,"kb_used_meta":42304,"kb_avail":94324952,"statfs":{"total":96636764160,"available":96588750848,"internally_reserved":0,"allocated":4677632,"data_stored":46553540,"data_compressed":2177252,"data_compressed_allocated":2867200,"data_compressed_original":45793280,"omap_allocated":0,"internal_metadata":43319296},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2265088,"data_stored":36175896,"data_compressed":1717824,"data_compressed_allocated":2260992,"data_compressed_original":36175872,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8343552,"data_stored":133431320,"data_compressed":6336032,"data_compressed_allocated":8339456,"data_compressed_original":133431296,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1052672,"data_stored":16842752,"data_compressed":799784,"data_compressed_allocated":1052672,"data_compressed_original":16842752,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":15400984,"data_compressed":731320,"data_compressed_allocated":962560,"data_compressed_original":15400960,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:55.541 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:55.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:55.665+0000 7f73d4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:55.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:55.671+0000 7f73d4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:56.041 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-22T03:28:56.041 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-22T03:28:56.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:56.150+0000 7ff0b602c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:56.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:56.166+0000 7ff0b602c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:57.162 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_3' removed 2024-09-22T03:28:57.181 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:28:57.181 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:28:57.181 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-22T03:28:57.182 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_4 16 2024-09-22T03:28:57.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:57.299+0000 7f0e2bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:57.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:57.311+0000 7f0e2bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:57.763 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:57.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:57.862+0000 7f789a557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:57.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:57.866+0000 7f789a557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.166 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_4' created 2024-09-22T03:28:58.185 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-22T03:28:58.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:58.290+0000 7fceb2957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:58.303+0000 7fceb2957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.355 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:58.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:58.458+0000 7f7ef6d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:58.467+0000 7f7ef6d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.644 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:58.645 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:28:58.657 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":752,"stamp":"2024-09-22T03:28:58.165929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214540,"kb_used_data":10124,"kb_used_omap":0,"kb_used_meta":204352,"kb_avail":377272820,"statfs":{"total":386547056640,"available":386327367680,"internally_reserved":0,"allocated":10366976,"data_stored":50748463,"data_compressed":2271428,"data_compressed_allocated":2994176,"data_compressed_original":47661056,"omap_allocated":0,"internal_metadata":209256448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-32112664,"num_objects":-491,"num_object_clones":0,"num_object_copies":-1473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-35557,"num_write_kb":-1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3724,"num_bytes_recovered":-1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.632869"},"pg_stats":[{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":438,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.191813+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:28:57.191813+0000","last_peered":"2024-09-22T03:28:57.191813+0000","last_clean":"2024-09-22T03:28:57.191813+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:28:57.191813+0000","last_undegraded":"2024-09-22T03:28:57.191813+0000","last_fullsized":"2024-09-22T03:28:57.191813+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":496,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.168561+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:28:56.168561+0000","last_peered":"2024-09-22T03:28:56.168561+0000","last_clean":"2024-09-22T03:28:56.168561+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:28:56.168561+0000","last_undegraded":"2024-09-22T03:28:56.168561+0000","last_fullsized":"2024-09-22T03:28:56.168561+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":563,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.186956+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:28:57.186956+0000","last_peered":"2024-09-22T03:28:57.186956+0000","last_clean":"2024-09-22T03:28:57.186956+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:28:57.186956+0000","last_undegraded":"2024-09-22T03:28:57.186956+0000","last_fullsized":"2024-09-22T03:28:57.186956+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":529,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.191444+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:28:57.191444+0000","last_peered":"2024-09-22T03:28:57.191444+0000","last_clean":"2024-09-22T03:28:57.191444+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:28:57.191444+0000","last_undegraded":"2024-09-22T03:28:57.191444+0000","last_fullsized":"2024-09-22T03:28:57.191444+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":613,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.169936+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:28:56.169936+0000","last_peered":"2024-09-22T03:28:56.169936+0000","last_clean":"2024-09-22T03:28:56.169936+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:28:56.169936+0000","last_undegraded":"2024-09-22T03:28:56.169936+0000","last_fullsized":"2024-09-22T03:28:56.169936+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":330,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.179911+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:28:56.179911+0000","last_peered":"2024-09-22T03:28:56.179911+0000","last_clean":"2024-09-22T03:28:56.179911+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:28:56.179911+0000","last_undegraded":"2024-09-22T03:28:56.179911+0000","last_fullsized":"2024-09-22T03:28:56.179911+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":457,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.172257+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:28:57.172257+0000","last_peered":"2024-09-22T03:28:57.172257+0000","last_clean":"2024-09-22T03:28:57.172257+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:28:57.172257+0000","last_undegraded":"2024-09-22T03:28:57.172257+0000","last_fullsized":"2024-09-22T03:28:57.172257+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":401,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.181287+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:28:56.181287+0000","last_peered":"2024-09-22T03:28:56.181287+0000","last_clean":"2024-09-22T03:28:56.181287+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:28:56.181287+0000","last_undegraded":"2024-09-22T03:28:56.181287+0000","last_fullsized":"2024-09-22T03:28:56.181287+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":407,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.178037+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:28:56.178037+0000","last_peered":"2024-09-22T03:28:56.178037+0000","last_clean":"2024-09-22T03:28:56.178037+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:28:56.178037+0000","last_undegraded":"2024-09-22T03:28:56.178037+0000","last_fullsized":"2024-09-22T03:28:56.178037+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599307,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88144,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":86336,"kb_avail":94283696,"statfs":{"total":96636764160,"available":96546504704,"internally_reserved":0,"allocated":1835008,"data_stored":760866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":88408064},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":39628,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94332212,"statfs":{"total":96636764160,"available":96596185088,"internally_reserved":0,"allocated":1896448,"data_stored":1351234,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38452,"kb_used_data":4644,"kb_used_omap":0,"kb_used_meta":33792,"kb_avail":94333388,"statfs":{"total":96636764160,"available":96597389312,"internally_reserved":0,"allocated":4755456,"data_stored":47288559,"data_compressed":2211484,"data_compressed_allocated":2912256,"data_compressed_original":46514176,"omap_allocated":0,"internal_metadata":34603008},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48316,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94323524,"statfs":{"total":96636764160,"available":96587288576,"internally_reserved":0,"allocated":1880064,"data_stored":1347804,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:58.658 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:28:58.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:58.756+0000 7f3867157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:58.761+0000 7f3867157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.005 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:59.006 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":1083,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":222,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975721,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9}],"num_pgs":9,"num_pools":2,"num_objects":4,"data_bytes":590387,"bytes_used":235196416,"bytes_avail":386311860224,"bytes_total":386547056640},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T03:27:01.429211+0000","services":{}},"progress_events":{}} 2024-09-22T03:28:59.019 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-22T03:28:59.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:59.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:59.119+0000 7f9d13903640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:28:59.126+0000 7f9d13903640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.167 INFO:teuthology.orchestra.run.smithi146.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-22T03:28:59.381 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:28:59.382 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:28:59.394 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":753,"stamp":"2024-09-22T03:28:59.169050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214540,"kb_used_data":10124,"kb_used_omap":0,"kb_used_meta":204352,"kb_avail":377272820,"statfs":{"total":386547056640,"available":386327367680,"internally_reserved":0,"allocated":10366976,"data_stored":50748463,"data_compressed":2271428,"data_compressed_allocated":2994176,"data_compressed_original":47661056,"omap_allocated":0,"internal_metadata":209256448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-32112664,"num_objects":-491,"num_object_clones":0,"num_object_copies":-1473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-35557,"num_write_kb":-1153473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3724,"num_bytes_recovered":-1310744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015953"},"pg_stats":[{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":438,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.191813+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:28:57.191813+0000","last_peered":"2024-09-22T03:28:57.191813+0000","last_clean":"2024-09-22T03:28:57.191813+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:28:57.191813+0000","last_undegraded":"2024-09-22T03:28:57.191813+0000","last_fullsized":"2024-09-22T03:28:57.191813+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":496,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.168561+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:28:56.168561+0000","last_peered":"2024-09-22T03:28:56.168561+0000","last_clean":"2024-09-22T03:28:56.168561+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:28:56.168561+0000","last_undegraded":"2024-09-22T03:28:56.168561+0000","last_fullsized":"2024-09-22T03:28:56.168561+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":563,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.186956+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:28:57.186956+0000","last_peered":"2024-09-22T03:28:57.186956+0000","last_clean":"2024-09-22T03:28:57.186956+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:28:57.186956+0000","last_undegraded":"2024-09-22T03:28:57.186956+0000","last_fullsized":"2024-09-22T03:28:57.186956+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":529,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.191444+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:28:57.191444+0000","last_peered":"2024-09-22T03:28:57.191444+0000","last_clean":"2024-09-22T03:28:57.191444+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:28:57.191444+0000","last_undegraded":"2024-09-22T03:28:57.191444+0000","last_fullsized":"2024-09-22T03:28:57.191444+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":613,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.169936+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:28:56.169936+0000","last_peered":"2024-09-22T03:28:56.169936+0000","last_clean":"2024-09-22T03:28:56.169936+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:28:56.169936+0000","last_undegraded":"2024-09-22T03:28:56.169936+0000","last_fullsized":"2024-09-22T03:28:56.169936+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":330,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.179911+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:28:56.179911+0000","last_peered":"2024-09-22T03:28:56.179911+0000","last_clean":"2024-09-22T03:28:56.179911+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:28:56.179911+0000","last_undegraded":"2024-09-22T03:28:56.179911+0000","last_fullsized":"2024-09-22T03:28:56.179911+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":457,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.172257+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:28:57.172257+0000","last_peered":"2024-09-22T03:28:57.172257+0000","last_clean":"2024-09-22T03:28:57.172257+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:28:57.172257+0000","last_undegraded":"2024-09-22T03:28:57.172257+0000","last_fullsized":"2024-09-22T03:28:57.172257+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":401,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.181287+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:28:56.181287+0000","last_peered":"2024-09-22T03:28:56.181287+0000","last_clean":"2024-09-22T03:28:56.181287+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:28:56.181287+0000","last_undegraded":"2024-09-22T03:28:56.181287+0000","last_fullsized":"2024-09-22T03:28:56.181287+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":407,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.178037+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:28:56.178037+0000","last_peered":"2024-09-22T03:28:56.178037+0000","last_clean":"2024-09-22T03:28:56.178037+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:28:56.178037+0000","last_undegraded":"2024-09-22T03:28:56.178037+0000","last_fullsized":"2024-09-22T03:28:56.178037+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599307,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88144,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":86336,"kb_avail":94283696,"statfs":{"total":96636764160,"available":96546504704,"internally_reserved":0,"allocated":1835008,"data_stored":760866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":88408064},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":39628,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94332212,"statfs":{"total":96636764160,"available":96596185088,"internally_reserved":0,"allocated":1896448,"data_stored":1351234,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38452,"kb_used_data":4644,"kb_used_omap":0,"kb_used_meta":33792,"kb_avail":94333388,"statfs":{"total":96636764160,"available":96597389312,"internally_reserved":0,"allocated":4755456,"data_stored":47288559,"data_compressed":2211484,"data_compressed_allocated":2912256,"data_compressed_original":46514176,"omap_allocated":0,"internal_metadata":34603008},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48316,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94323524,"statfs":{"total":96636764160,"available":96587288576,"internally_reserved":0,"allocated":1880064,"data_stored":1347804,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:59.936 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:00.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:00.040+0000 7f1192903640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:00.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:00.044+0000 7f1192903640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:00.186 DEBUG:teuthology.orchestra.run.smithi146:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-22T03:29:00.240 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:29:00.240 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:29:00.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:hints = 1 2024-09-22T03:29:00.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:29:00.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object prefix: benchmark_data_smithi146_54403 2024-09-22T03:29:00.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:29:00.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:29:01.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 1 16 350 334 20.8738 20.875 0.0188847 0.0469692 2024-09-22T03:29:02.116 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:02.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:02.243+0000 7f6dae517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:02.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:02.250+0000 7f6dae517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:02.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 2 16 683 667 20.8422 20.8125 0.0824475 0.0473791 2024-09-22T03:29:02.396 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:02.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:02.526+0000 7f40d442c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:02.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:02.536+0000 7f40d442c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:02.840 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:29:02.841 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:29:02.861 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":756,"stamp":"2024-09-22T03:29:01.454360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6750771,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":233,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":190,"ondisk_log_size":190,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":217028,"kb_used_data":8452,"kb_used_omap":0,"kb_used_meta":208512,"kb_avail":377270332,"statfs":{"total":386547056640,"available":386324819968,"internally_reserved":0,"allocated":8654848,"data_stored":22778848,"data_compressed":942242,"data_compressed_allocated":1245184,"data_compressed_original":19677184,"omap_allocated":0,"internal_metadata":213516288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.291975"},"pg_stats":[{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"224'21","reported_seq":34,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.176691+0000","last_change":"2024-09-22T03:28:59.274720+0000","last_active":"2024-09-22T03:29:01.176691+0000","last_peered":"2024-09-22T03:29:01.176691+0000","last_clean":"2024-09-22T03:29:01.176691+0000","last_became_active":"2024-09-22T03:28:59.242935+0000","last_became_peered":"2024-09-22T03:28:59.242935+0000","last_unstale":"2024-09-22T03:29:01.176691+0000","last_undegraded":"2024-09-22T03:29:01.176691+0000","last_fullsized":"2024-09-22T03:29:01.176691+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"224'22","reported_seq":35,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.265044+0000","last_change":"2024-09-22T03:28:59.269315+0000","last_active":"2024-09-22T03:29:01.265044+0000","last_peered":"2024-09-22T03:29:01.265044+0000","last_clean":"2024-09-22T03:29:01.265044+0000","last_became_active":"2024-09-22T03:28:59.237641+0000","last_became_peered":"2024-09-22T03:28:59.237641+0000","last_unstale":"2024-09-22T03:29:01.265044+0000","last_undegraded":"2024-09-22T03:29:01.265044+0000","last_fullsized":"2024-09-22T03:29:01.265044+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":438,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.191813+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:28:57.191813+0000","last_peered":"2024-09-22T03:28:57.191813+0000","last_clean":"2024-09-22T03:28:57.191813+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:28:57.191813+0000","last_undegraded":"2024-09-22T03:28:57.191813+0000","last_fullsized":"2024-09-22T03:28:57.191813+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"224'26","reported_seq":39,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.291241+0000","last_change":"2024-09-22T03:28:59.285414+0000","last_active":"2024-09-22T03:29:01.291241+0000","last_peered":"2024-09-22T03:29:01.291241+0000","last_clean":"2024-09-22T03:29:01.291241+0000","last_became_active":"2024-09-22T03:28:59.276992+0000","last_became_peered":"2024-09-22T03:28:59.276992+0000","last_unstale":"2024-09-22T03:29:01.291241+0000","last_undegraded":"2024-09-22T03:29:01.291241+0000","last_fullsized":"2024-09-22T03:29:01.291241+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":496,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.168561+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:28:56.168561+0000","last_peered":"2024-09-22T03:28:56.168561+0000","last_clean":"2024-09-22T03:28:56.168561+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:28:56.168561+0000","last_undegraded":"2024-09-22T03:28:56.168561+0000","last_fullsized":"2024-09-22T03:28:56.168561+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":563,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.186956+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:28:57.186956+0000","last_peered":"2024-09-22T03:28:57.186956+0000","last_clean":"2024-09-22T03:28:57.186956+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:28:57.186956+0000","last_undegraded":"2024-09-22T03:28:57.186956+0000","last_fullsized":"2024-09-22T03:28:57.186956+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"224'21","reported_seq":34,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.315845+0000","last_change":"2024-09-22T03:28:59.289307+0000","last_active":"2024-09-22T03:29:01.315845+0000","last_peered":"2024-09-22T03:29:01.315845+0000","last_clean":"2024-09-22T03:29:01.315845+0000","last_became_active":"2024-09-22T03:28:59.282573+0000","last_became_peered":"2024-09-22T03:28:59.282573+0000","last_unstale":"2024-09-22T03:29:01.315845+0000","last_undegraded":"2024-09-22T03:29:01.315845+0000","last_fullsized":"2024-09-22T03:29:01.315845+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":529,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.191444+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:28:57.191444+0000","last_peered":"2024-09-22T03:28:57.191444+0000","last_clean":"2024-09-22T03:28:57.191444+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:28:57.191444+0000","last_undegraded":"2024-09-22T03:28:57.191444+0000","last_fullsized":"2024-09-22T03:28:57.191444+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":613,"reported_epoch":220,"state":"active+clean","last_fresh":"2024-09-22T03:28:56.169936+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:28:56.169936+0000","last_peered":"2024-09-22T03:28:56.169936+0000","last_clean":"2024-09-22T03:28:56.169936+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:28:56.169936+0000","last_undegraded":"2024-09-22T03:28:56.169936+0000","last_fullsized":"2024-09-22T03:28:56.169936+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":342,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.160849+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:29:00.160849+0000","last_peered":"2024-09-22T03:29:00.160849+0000","last_clean":"2024-09-22T03:29:00.160849+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:29:00.160849+0000","last_undegraded":"2024-09-22T03:29:00.160849+0000","last_fullsized":"2024-09-22T03:29:00.160849+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":457,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-22T03:28:57.172257+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:28:57.172257+0000","last_peered":"2024-09-22T03:28:57.172257+0000","last_clean":"2024-09-22T03:28:57.172257+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:28:57.172257+0000","last_undegraded":"2024-09-22T03:28:57.172257+0000","last_fullsized":"2024-09-22T03:28:57.172257+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":413,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.160625+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:29:00.160625+0000","last_peered":"2024-09-22T03:29:00.160625+0000","last_clean":"2024-09-22T03:29:00.160625+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:29:00.160625+0000","last_undegraded":"2024-09-22T03:29:00.160625+0000","last_fullsized":"2024-09-22T03:29:00.160625+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:28:58.162328+0000","last_change":"2024-09-22T03:28:58.162328+0000","last_active":"2024-09-22T03:28:58.162328+0000","last_peered":"2024-09-22T03:28:58.162328+0000","last_clean":"2024-09-22T03:28:58.162328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:58.162328+0000","last_undegraded":"2024-09-22T03:28:58.162328+0000","last_fullsized":"2024-09-22T03:28:58.162328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":419,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.156659+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:29:00.156659+0000","last_peered":"2024-09-22T03:29:00.156659+0000","last_clean":"2024-09-22T03:29:00.156659+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:29:00.156659+0000","last_undegraded":"2024-09-22T03:29:00.156659+0000","last_fullsized":"2024-09-22T03:29:00.156659+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1208320,"data_stored":19333120,"data_compressed":917657,"data_compressed_allocated":1208320,"data_compressed_original":19333120,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599307,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88144,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":86336,"kb_avail":94283696,"statfs":{"total":96636764160,"available":96546504704,"internally_reserved":0,"allocated":1835008,"data_stored":760866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":88408064},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":39628,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94332212,"statfs":{"total":96636764160,"available":96596185088,"internally_reserved":0,"allocated":1896448,"data_stored":1351234,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40940,"kb_used_data":2972,"kb_used_omap":0,"kb_used_meta":37952,"kb_avail":94330900,"statfs":{"total":96636764160,"available":96594841600,"internally_reserved":0,"allocated":3043328,"data_stored":19318944,"data_compressed":882298,"data_compressed_allocated":1163264,"data_compressed_original":18530304,"omap_allocated":0,"internal_metadata":38862848},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48316,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94323524,"statfs":{"total":96636764160,"available":96587288576,"internally_reserved":0,"allocated":1880064,"data_stored":1347804,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1208320,"data_stored":19333120,"data_compressed":917657,"data_compressed_allocated":1208320,"data_compressed_original":19333120,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:02.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:29:03.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:03.007+0000 7f27e28ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:03.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:03.014+0000 7f27e28ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:03.295 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 3 16 1025 1009 21.0193 21.375 0.0861057 0.0470025 2024-09-22T03:29:03.462 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:29:03.462 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":1088,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":224,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975721,"num_in_osds":4,"osd_in_since":1726974657,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"unknown","count":12}],"num_pgs":25,"num_pools":3,"num_objects":98,"data_bytes":6750771,"bytes_used":222236672,"bytes_avail":386324819968,"bytes_total":386547056640,"unknown_pgs_ratio":0.47999998927116394,"write_bytes_sec":1435325,"read_op_per_sec":0,"write_op_per_sec":21},"fsmap":{"epoch":1,"btime":"2024-09-22T03:10:55:007935+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T03:27:01.429211+0000","services":{}},"progress_events":{}} 2024-09-22T03:29:03.481 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-22T03:29:03.482 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:03.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:03.625+0000 7f3c9aa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:03.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:03.635+0000 7f3c9aa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:29:03.891 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:29:03.906 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":757,"stamp":"2024-09-22T03:29:03.454737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51511859,"num_objects":781,"num_object_clones":0,"num_object_copies":2343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":916,"num_write_kb":51624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":248740,"kb_used_data":13792,"kb_used_omap":0,"kb_used_meta":234880,"kb_avail":377238620,"statfs":{"total":386547056640,"available":386292346880,"internally_reserved":0,"allocated":14123008,"data_stored":108929101,"data_compressed":5029854,"data_compressed_allocated":6627328,"data_compressed_original":105791488,"omap_allocated":0,"internal_metadata":240517120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50921472,"num_objects":777,"num_object_clones":0,"num_object_copies":2331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":49728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000966"},"pg_stats":[{"pgid":"7.f","version":"224'54","reported_seq":67,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.012453+0000","last_change":"2024-09-22T03:28:59.304274+0000","last_active":"2024-09-22T03:29:03.012453+0000","last_peered":"2024-09-22T03:29:03.012453+0000","last_clean":"2024-09-22T03:29:03.012453+0000","last_became_active":"2024-09-22T03:28:59.301139+0000","last_became_peered":"2024-09-22T03:28:59.301139+0000","last_unstale":"2024-09-22T03:29:03.012453+0000","last_undegraded":"2024-09-22T03:29:03.012453+0000","last_fullsized":"2024-09-22T03:29:03.012453+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"224'21","reported_seq":34,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.176691+0000","last_change":"2024-09-22T03:28:59.274720+0000","last_active":"2024-09-22T03:29:01.176691+0000","last_peered":"2024-09-22T03:29:01.176691+0000","last_clean":"2024-09-22T03:29:01.176691+0000","last_became_active":"2024-09-22T03:28:59.242935+0000","last_became_peered":"2024-09-22T03:28:59.242935+0000","last_unstale":"2024-09-22T03:29:01.176691+0000","last_undegraded":"2024-09-22T03:29:01.176691+0000","last_fullsized":"2024-09-22T03:29:01.176691+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"224'58","reported_seq":71,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.851868+0000","last_change":"2024-09-22T03:28:59.253687+0000","last_active":"2024-09-22T03:29:02.851868+0000","last_peered":"2024-09-22T03:29:02.851868+0000","last_clean":"2024-09-22T03:29:02.851868+0000","last_became_active":"2024-09-22T03:28:59.248951+0000","last_became_peered":"2024-09-22T03:28:59.248951+0000","last_unstale":"2024-09-22T03:29:02.851868+0000","last_undegraded":"2024-09-22T03:29:02.851868+0000","last_fullsized":"2024-09-22T03:29:02.851868+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"224'45","reported_seq":58,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.833333+0000","last_change":"2024-09-22T03:28:59.245472+0000","last_active":"2024-09-22T03:29:02.833333+0000","last_peered":"2024-09-22T03:29:02.833333+0000","last_clean":"2024-09-22T03:29:02.833333+0000","last_became_active":"2024-09-22T03:28:59.242013+0000","last_became_peered":"2024-09-22T03:28:59.242013+0000","last_unstale":"2024-09-22T03:29:02.833333+0000","last_undegraded":"2024-09-22T03:29:02.833333+0000","last_fullsized":"2024-09-22T03:29:02.833333+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"224'67","reported_seq":80,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.029567+0000","last_change":"2024-09-22T03:28:59.239966+0000","last_active":"2024-09-22T03:29:03.029567+0000","last_peered":"2024-09-22T03:29:03.029567+0000","last_clean":"2024-09-22T03:29:03.029567+0000","last_became_active":"2024-09-22T03:28:59.234011+0000","last_became_peered":"2024-09-22T03:28:59.234011+0000","last_unstale":"2024-09-22T03:29:03.029567+0000","last_undegraded":"2024-09-22T03:29:03.029567+0000","last_fullsized":"2024-09-22T03:29:03.029567+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"224'67","reported_seq":80,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.096726+0000","last_change":"2024-09-22T03:28:59.254864+0000","last_active":"2024-09-22T03:29:03.096726+0000","last_peered":"2024-09-22T03:29:03.096726+0000","last_clean":"2024-09-22T03:29:03.096726+0000","last_became_active":"2024-09-22T03:28:59.246725+0000","last_became_peered":"2024-09-22T03:28:59.246725+0000","last_unstale":"2024-09-22T03:29:03.096726+0000","last_undegraded":"2024-09-22T03:29:03.096726+0000","last_fullsized":"2024-09-22T03:29:03.096726+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"224'57","reported_seq":70,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.907652+0000","last_change":"2024-09-22T03:28:59.252446+0000","last_active":"2024-09-22T03:29:02.907652+0000","last_peered":"2024-09-22T03:29:02.907652+0000","last_clean":"2024-09-22T03:29:02.907652+0000","last_became_active":"2024-09-22T03:28:59.247927+0000","last_became_peered":"2024-09-22T03:28:59.247927+0000","last_unstale":"2024-09-22T03:29:02.907652+0000","last_undegraded":"2024-09-22T03:29:02.907652+0000","last_fullsized":"2024-09-22T03:29:02.907652+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"224'22","reported_seq":35,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.265044+0000","last_change":"2024-09-22T03:28:59.269315+0000","last_active":"2024-09-22T03:29:01.265044+0000","last_peered":"2024-09-22T03:29:01.265044+0000","last_clean":"2024-09-22T03:29:01.265044+0000","last_became_active":"2024-09-22T03:28:59.237641+0000","last_became_peered":"2024-09-22T03:28:59.237641+0000","last_unstale":"2024-09-22T03:29:01.265044+0000","last_undegraded":"2024-09-22T03:29:01.265044+0000","last_fullsized":"2024-09-22T03:29:01.265044+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"224'68","reported_seq":81,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.123777+0000","last_change":"2024-09-22T03:28:59.263362+0000","last_active":"2024-09-22T03:29:03.123777+0000","last_peered":"2024-09-22T03:29:03.123777+0000","last_clean":"2024-09-22T03:29:03.123777+0000","last_became_active":"2024-09-22T03:28:59.259464+0000","last_became_peered":"2024-09-22T03:28:59.259464+0000","last_unstale":"2024-09-22T03:29:03.123777+0000","last_undegraded":"2024-09-22T03:29:03.123777+0000","last_fullsized":"2024-09-22T03:29:03.123777+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":447,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.163399+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:29:00.163399+0000","last_peered":"2024-09-22T03:29:00.163399+0000","last_clean":"2024-09-22T03:29:00.163399+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:29:00.163399+0000","last_undegraded":"2024-09-22T03:29:00.163399+0000","last_fullsized":"2024-09-22T03:29:00.163399+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"224'26","reported_seq":39,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.291241+0000","last_change":"2024-09-22T03:28:59.285414+0000","last_active":"2024-09-22T03:29:01.291241+0000","last_peered":"2024-09-22T03:29:01.291241+0000","last_clean":"2024-09-22T03:29:01.291241+0000","last_became_active":"2024-09-22T03:28:59.276992+0000","last_became_peered":"2024-09-22T03:28:59.276992+0000","last_unstale":"2024-09-22T03:29:01.291241+0000","last_undegraded":"2024-09-22T03:29:01.291241+0000","last_fullsized":"2024-09-22T03:29:01.291241+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":508,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.156441+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:29:00.156441+0000","last_peered":"2024-09-22T03:29:00.156441+0000","last_clean":"2024-09-22T03:29:00.156441+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:29:00.156441+0000","last_undegraded":"2024-09-22T03:29:00.156441+0000","last_fullsized":"2024-09-22T03:29:00.156441+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"224'53","reported_seq":66,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.978438+0000","last_change":"2024-09-22T03:28:59.274177+0000","last_active":"2024-09-22T03:29:02.978438+0000","last_peered":"2024-09-22T03:29:02.978438+0000","last_clean":"2024-09-22T03:29:02.978438+0000","last_became_active":"2024-09-22T03:28:59.270710+0000","last_became_peered":"2024-09-22T03:28:59.270710+0000","last_unstale":"2024-09-22T03:29:02.978438+0000","last_undegraded":"2024-09-22T03:29:02.978438+0000","last_fullsized":"2024-09-22T03:29:02.978438+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":572,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.163352+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:29:00.163352+0000","last_peered":"2024-09-22T03:29:00.163352+0000","last_clean":"2024-09-22T03:29:00.163352+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:29:00.163352+0000","last_undegraded":"2024-09-22T03:29:00.163352+0000","last_fullsized":"2024-09-22T03:29:00.163352+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"224'21","reported_seq":34,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.315845+0000","last_change":"2024-09-22T03:28:59.289307+0000","last_active":"2024-09-22T03:29:01.315845+0000","last_peered":"2024-09-22T03:29:01.315845+0000","last_clean":"2024-09-22T03:29:01.315845+0000","last_became_active":"2024-09-22T03:28:59.282573+0000","last_became_peered":"2024-09-22T03:28:59.282573+0000","last_unstale":"2024-09-22T03:29:01.315845+0000","last_undegraded":"2024-09-22T03:29:01.315845+0000","last_fullsized":"2024-09-22T03:29:01.315845+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":538,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.161246+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:29:00.161246+0000","last_peered":"2024-09-22T03:29:00.161246+0000","last_clean":"2024-09-22T03:29:00.161246+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:29:00.161246+0000","last_undegraded":"2024-09-22T03:29:00.161246+0000","last_fullsized":"2024-09-22T03:29:00.161246+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":625,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.158162+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:29:00.158162+0000","last_peered":"2024-09-22T03:29:00.158162+0000","last_clean":"2024-09-22T03:29:00.158162+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:29:00.158162+0000","last_undegraded":"2024-09-22T03:29:00.158162+0000","last_fullsized":"2024-09-22T03:29:00.158162+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"224'63","reported_seq":76,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.109397+0000","last_change":"2024-09-22T03:28:59.258680+0000","last_active":"2024-09-22T03:29:03.109397+0000","last_peered":"2024-09-22T03:29:03.109397+0000","last_clean":"2024-09-22T03:29:03.109397+0000","last_became_active":"2024-09-22T03:28:59.251053+0000","last_became_peered":"2024-09-22T03:28:59.251053+0000","last_unstale":"2024-09-22T03:29:03.109397+0000","last_undegraded":"2024-09-22T03:29:03.109397+0000","last_fullsized":"2024-09-22T03:29:03.109397+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":342,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.160849+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:29:00.160849+0000","last_peered":"2024-09-22T03:29:00.160849+0000","last_clean":"2024-09-22T03:29:00.160849+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:29:00.160849+0000","last_undegraded":"2024-09-22T03:29:00.160849+0000","last_fullsized":"2024-09-22T03:29:00.160849+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"224'67","reported_seq":80,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.050147+0000","last_change":"2024-09-22T03:28:59.268256+0000","last_active":"2024-09-22T03:29:03.050147+0000","last_peered":"2024-09-22T03:29:03.050147+0000","last_clean":"2024-09-22T03:29:03.050147+0000","last_became_active":"2024-09-22T03:28:59.263256+0000","last_became_peered":"2024-09-22T03:28:59.263256+0000","last_unstale":"2024-09-22T03:29:03.050147+0000","last_undegraded":"2024-09-22T03:29:03.050147+0000","last_fullsized":"2024-09-22T03:29:03.050147+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":466,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.159043+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:29:00.159043+0000","last_peered":"2024-09-22T03:29:00.159043+0000","last_clean":"2024-09-22T03:29:00.159043+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:29:00.159043+0000","last_undegraded":"2024-09-22T03:29:00.159043+0000","last_fullsized":"2024-09-22T03:29:00.159043+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"224'41","reported_seq":54,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.925495+0000","last_change":"2024-09-22T03:28:59.267581+0000","last_active":"2024-09-22T03:29:02.925495+0000","last_peered":"2024-09-22T03:29:02.925495+0000","last_clean":"2024-09-22T03:29:02.925495+0000","last_became_active":"2024-09-22T03:28:59.261143+0000","last_became_peered":"2024-09-22T03:28:59.261143+0000","last_unstale":"2024-09-22T03:29:02.925495+0000","last_undegraded":"2024-09-22T03:29:02.925495+0000","last_fullsized":"2024-09-22T03:29:02.925495+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":413,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.160625+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:29:00.160625+0000","last_peered":"2024-09-22T03:29:00.160625+0000","last_clean":"2024-09-22T03:29:00.160625+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:29:00.160625+0000","last_undegraded":"2024-09-22T03:29:00.160625+0000","last_fullsized":"2024-09-22T03:29:00.160625+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"224'31","reported_seq":44,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.683093+0000","last_change":"2024-09-22T03:28:59.278638+0000","last_active":"2024-09-22T03:29:01.683093+0000","last_peered":"2024-09-22T03:29:01.683093+0000","last_clean":"2024-09-22T03:29:01.683093+0000","last_became_active":"2024-09-22T03:28:59.274902+0000","last_became_peered":"2024-09-22T03:28:59.274902+0000","last_unstale":"2024-09-22T03:29:01.683093+0000","last_undegraded":"2024-09-22T03:29:01.683093+0000","last_fullsized":"2024-09-22T03:29:01.683093+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":419,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.156659+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:29:00.156659+0000","last_peered":"2024-09-22T03:29:00.156659+0000","last_clean":"2024-09-22T03:29:00.156659+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:29:00.156659+0000","last_undegraded":"2024-09-22T03:29:00.156659+0000","last_fullsized":"2024-09-22T03:29:00.156659+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":50921472,"num_objects":777,"num_object_clones":0,"num_object_copies":2331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":49728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8609792,"data_stored":137756672,"data_compressed":6538992,"data_compressed_allocated":8609792,"data_compressed_original":137756672,"omap_allocated":0,"internal_metadata":0},"log_size":761,"ondisk_log_size":761,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599312,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95128,"kb_used_data":4616,"kb_used_omap":0,"kb_used_meta":90496,"kb_avail":94276712,"statfs":{"total":96636764160,"available":96539353088,"internally_reserved":0,"allocated":4726784,"data_stored":46646912,"data_compressed":2177595,"data_compressed_allocated":2867200,"data_compressed_original":45875200,"omap_allocated":0,"internal_metadata":92667904},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63440,"kb_used_data":3452,"kb_used_omap":0,"kb_used_meta":59968,"kb_avail":94308400,"statfs":{"total":96636764160,"available":96571801600,"internally_reserved":0,"allocated":3534848,"data_stored":27183264,"data_compressed":1255649,"data_compressed_allocated":1654784,"data_compressed_original":26394624,"omap_allocated":0,"internal_metadata":61407232},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40940,"kb_used_data":2972,"kb_used_omap":0,"kb_used_meta":37952,"kb_avail":94330900,"statfs":{"total":96636764160,"available":96594841600,"internally_reserved":0,"allocated":3043328,"data_stored":19318944,"data_compressed":882298,"data_compressed_allocated":1163264,"data_compressed_original":18530304,"omap_allocated":0,"internal_metadata":38862848},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328449,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49232,"kb_used_data":2752,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94322608,"statfs":{"total":96636764160,"available":96586350592,"internally_reserved":0,"allocated":2818048,"data_stored":15779981,"data_compressed":714312,"data_compressed_allocated":942080,"data_compressed_original":14991360,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1552384,"data_stored":24838144,"data_compressed":1178989,"data_compressed_allocated":1552384,"data_compressed_original":24838144,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1208320,"data_stored":19333120,"data_compressed":917657,"data_compressed_allocated":1208320,"data_compressed_original":19333120,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":33030144,"data_compressed":1567887,"data_compressed_allocated":2064384,"data_compressed_original":33030144,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3784704,"data_stored":60555264,"data_compressed":2874459,"data_compressed_allocated":3784704,"data_compressed_original":60555264,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:04.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 4 16 1397 1381 21.5763 23.25 0.010246 0.045981 2024-09-22T03:29:04.438 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:04.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:04.573+0000 7f37da597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:04.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:04.583+0000 7f37da597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:05.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 5 16 1765 1749 21.8605 23 0.0376945 0.0454078 2024-09-22T03:29:06.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 6 16 2116 2100 21.8731 21.9375 0.0458354 0.0455021 2024-09-22T03:29:06.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:06.908 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:06.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:06.923+0000 7f7855957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:06.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:06.932+0000 7f7855957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:07.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:07.046+0000 7f63b9f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:07.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:07.056+0000 7f63b9f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:07.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 7 16 2464 2448 21.8553 21.75 0.0560792 0.0455212 2024-09-22T03:29:07.348 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:29:07.349 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:29:07.367 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":758,"stamp":"2024-09-22T03:29:05.455093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51511859,"num_objects":781,"num_object_clones":0,"num_object_copies":2343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":916,"num_write_kb":51624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":248740,"kb_used_data":13792,"kb_used_omap":0,"kb_used_meta":234880,"kb_avail":377238620,"statfs":{"total":386547056640,"available":386292346880,"internally_reserved":0,"allocated":14123008,"data_stored":108929101,"data_compressed":5029854,"data_compressed_allocated":6627328,"data_compressed_original":105791488,"omap_allocated":0,"internal_metadata":240517120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50921472,"num_objects":777,"num_object_clones":0,"num_object_copies":2331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":49728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.289164"},"pg_stats":[{"pgid":"7.f","version":"224'54","reported_seq":67,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.012453+0000","last_change":"2024-09-22T03:28:59.304274+0000","last_active":"2024-09-22T03:29:03.012453+0000","last_peered":"2024-09-22T03:29:03.012453+0000","last_clean":"2024-09-22T03:29:03.012453+0000","last_became_active":"2024-09-22T03:28:59.301139+0000","last_became_peered":"2024-09-22T03:28:59.301139+0000","last_unstale":"2024-09-22T03:29:03.012453+0000","last_undegraded":"2024-09-22T03:29:03.012453+0000","last_fullsized":"2024-09-22T03:29:03.012453+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"224'21","reported_seq":34,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.176691+0000","last_change":"2024-09-22T03:28:59.274720+0000","last_active":"2024-09-22T03:29:01.176691+0000","last_peered":"2024-09-22T03:29:01.176691+0000","last_clean":"2024-09-22T03:29:01.176691+0000","last_became_active":"2024-09-22T03:28:59.242935+0000","last_became_peered":"2024-09-22T03:28:59.242935+0000","last_unstale":"2024-09-22T03:29:01.176691+0000","last_undegraded":"2024-09-22T03:29:01.176691+0000","last_fullsized":"2024-09-22T03:29:01.176691+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"224'58","reported_seq":71,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.851868+0000","last_change":"2024-09-22T03:28:59.253687+0000","last_active":"2024-09-22T03:29:02.851868+0000","last_peered":"2024-09-22T03:29:02.851868+0000","last_clean":"2024-09-22T03:29:02.851868+0000","last_became_active":"2024-09-22T03:28:59.248951+0000","last_became_peered":"2024-09-22T03:28:59.248951+0000","last_unstale":"2024-09-22T03:29:02.851868+0000","last_undegraded":"2024-09-22T03:29:02.851868+0000","last_fullsized":"2024-09-22T03:29:02.851868+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"224'45","reported_seq":58,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.833333+0000","last_change":"2024-09-22T03:28:59.245472+0000","last_active":"2024-09-22T03:29:02.833333+0000","last_peered":"2024-09-22T03:29:02.833333+0000","last_clean":"2024-09-22T03:29:02.833333+0000","last_became_active":"2024-09-22T03:28:59.242013+0000","last_became_peered":"2024-09-22T03:28:59.242013+0000","last_unstale":"2024-09-22T03:29:02.833333+0000","last_undegraded":"2024-09-22T03:29:02.833333+0000","last_fullsized":"2024-09-22T03:29:02.833333+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"224'67","reported_seq":80,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.029567+0000","last_change":"2024-09-22T03:28:59.239966+0000","last_active":"2024-09-22T03:29:03.029567+0000","last_peered":"2024-09-22T03:29:03.029567+0000","last_clean":"2024-09-22T03:29:03.029567+0000","last_became_active":"2024-09-22T03:28:59.234011+0000","last_became_peered":"2024-09-22T03:28:59.234011+0000","last_unstale":"2024-09-22T03:29:03.029567+0000","last_undegraded":"2024-09-22T03:29:03.029567+0000","last_fullsized":"2024-09-22T03:29:03.029567+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"224'67","reported_seq":80,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.096726+0000","last_change":"2024-09-22T03:28:59.254864+0000","last_active":"2024-09-22T03:29:03.096726+0000","last_peered":"2024-09-22T03:29:03.096726+0000","last_clean":"2024-09-22T03:29:03.096726+0000","last_became_active":"2024-09-22T03:28:59.246725+0000","last_became_peered":"2024-09-22T03:28:59.246725+0000","last_unstale":"2024-09-22T03:29:03.096726+0000","last_undegraded":"2024-09-22T03:29:03.096726+0000","last_fullsized":"2024-09-22T03:29:03.096726+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"224'57","reported_seq":70,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.907652+0000","last_change":"2024-09-22T03:28:59.252446+0000","last_active":"2024-09-22T03:29:02.907652+0000","last_peered":"2024-09-22T03:29:02.907652+0000","last_clean":"2024-09-22T03:29:02.907652+0000","last_became_active":"2024-09-22T03:28:59.247927+0000","last_became_peered":"2024-09-22T03:28:59.247927+0000","last_unstale":"2024-09-22T03:29:02.907652+0000","last_undegraded":"2024-09-22T03:29:02.907652+0000","last_fullsized":"2024-09-22T03:29:02.907652+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"224'22","reported_seq":35,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.265044+0000","last_change":"2024-09-22T03:28:59.269315+0000","last_active":"2024-09-22T03:29:01.265044+0000","last_peered":"2024-09-22T03:29:01.265044+0000","last_clean":"2024-09-22T03:29:01.265044+0000","last_became_active":"2024-09-22T03:28:59.237641+0000","last_became_peered":"2024-09-22T03:28:59.237641+0000","last_unstale":"2024-09-22T03:29:01.265044+0000","last_undegraded":"2024-09-22T03:29:01.265044+0000","last_fullsized":"2024-09-22T03:29:01.265044+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"224'68","reported_seq":81,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.123777+0000","last_change":"2024-09-22T03:28:59.263362+0000","last_active":"2024-09-22T03:29:03.123777+0000","last_peered":"2024-09-22T03:29:03.123777+0000","last_clean":"2024-09-22T03:29:03.123777+0000","last_became_active":"2024-09-22T03:28:59.259464+0000","last_became_peered":"2024-09-22T03:28:59.259464+0000","last_unstale":"2024-09-22T03:29:03.123777+0000","last_undegraded":"2024-09-22T03:29:03.123777+0000","last_fullsized":"2024-09-22T03:29:03.123777+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":447,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.163399+0000","last_change":"2024-09-22T03:28:00.178394+0000","last_active":"2024-09-22T03:29:00.163399+0000","last_peered":"2024-09-22T03:29:00.163399+0000","last_clean":"2024-09-22T03:29:00.163399+0000","last_became_active":"2024-09-22T03:27:57.289682+0000","last_became_peered":"2024-09-22T03:27:57.289682+0000","last_unstale":"2024-09-22T03:29:00.163399+0000","last_undegraded":"2024-09-22T03:29:00.163399+0000","last_fullsized":"2024-09-22T03:29:00.163399+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"224'26","reported_seq":39,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.291241+0000","last_change":"2024-09-22T03:28:59.285414+0000","last_active":"2024-09-22T03:29:01.291241+0000","last_peered":"2024-09-22T03:29:01.291241+0000","last_clean":"2024-09-22T03:29:01.291241+0000","last_became_active":"2024-09-22T03:28:59.276992+0000","last_became_peered":"2024-09-22T03:28:59.276992+0000","last_unstale":"2024-09-22T03:29:01.291241+0000","last_undegraded":"2024-09-22T03:29:01.291241+0000","last_fullsized":"2024-09-22T03:29:01.291241+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":508,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.156441+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:29:00.156441+0000","last_peered":"2024-09-22T03:29:00.156441+0000","last_clean":"2024-09-22T03:29:00.156441+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:29:00.156441+0000","last_undegraded":"2024-09-22T03:29:00.156441+0000","last_fullsized":"2024-09-22T03:29:00.156441+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"224'53","reported_seq":66,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.978438+0000","last_change":"2024-09-22T03:28:59.274177+0000","last_active":"2024-09-22T03:29:02.978438+0000","last_peered":"2024-09-22T03:29:02.978438+0000","last_clean":"2024-09-22T03:29:02.978438+0000","last_became_active":"2024-09-22T03:28:59.270710+0000","last_became_peered":"2024-09-22T03:28:59.270710+0000","last_unstale":"2024-09-22T03:29:02.978438+0000","last_undegraded":"2024-09-22T03:29:02.978438+0000","last_fullsized":"2024-09-22T03:29:02.978438+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":572,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.163352+0000","last_change":"2024-09-22T03:28:43.529144+0000","last_active":"2024-09-22T03:29:00.163352+0000","last_peered":"2024-09-22T03:29:00.163352+0000","last_clean":"2024-09-22T03:29:00.163352+0000","last_became_active":"2024-09-22T03:28:42.309830+0000","last_became_peered":"2024-09-22T03:28:42.309830+0000","last_unstale":"2024-09-22T03:29:00.163352+0000","last_undegraded":"2024-09-22T03:29:00.163352+0000","last_fullsized":"2024-09-22T03:29:00.163352+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"224'21","reported_seq":34,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.315845+0000","last_change":"2024-09-22T03:28:59.289307+0000","last_active":"2024-09-22T03:29:01.315845+0000","last_peered":"2024-09-22T03:29:01.315845+0000","last_clean":"2024-09-22T03:29:01.315845+0000","last_became_active":"2024-09-22T03:28:59.282573+0000","last_became_peered":"2024-09-22T03:28:59.282573+0000","last_unstale":"2024-09-22T03:29:01.315845+0000","last_undegraded":"2024-09-22T03:29:01.315845+0000","last_fullsized":"2024-09-22T03:29:01.315845+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":538,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.161246+0000","last_change":"2024-09-22T03:28:43.311040+0000","last_active":"2024-09-22T03:29:00.161246+0000","last_peered":"2024-09-22T03:29:00.161246+0000","last_clean":"2024-09-22T03:29:00.161246+0000","last_became_active":"2024-09-22T03:28:42.295474+0000","last_became_peered":"2024-09-22T03:28:42.295474+0000","last_unstale":"2024-09-22T03:29:00.161246+0000","last_undegraded":"2024-09-22T03:29:00.161246+0000","last_fullsized":"2024-09-22T03:29:00.161246+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":625,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.158162+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:29:00.158162+0000","last_peered":"2024-09-22T03:29:00.158162+0000","last_clean":"2024-09-22T03:29:00.158162+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:29:00.158162+0000","last_undegraded":"2024-09-22T03:29:00.158162+0000","last_fullsized":"2024-09-22T03:29:00.158162+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"224'63","reported_seq":76,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.109397+0000","last_change":"2024-09-22T03:28:59.258680+0000","last_active":"2024-09-22T03:29:03.109397+0000","last_peered":"2024-09-22T03:29:03.109397+0000","last_clean":"2024-09-22T03:29:03.109397+0000","last_became_active":"2024-09-22T03:28:59.251053+0000","last_became_peered":"2024-09-22T03:28:59.251053+0000","last_unstale":"2024-09-22T03:29:03.109397+0000","last_undegraded":"2024-09-22T03:29:03.109397+0000","last_fullsized":"2024-09-22T03:29:03.109397+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":342,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.160849+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:29:00.160849+0000","last_peered":"2024-09-22T03:29:00.160849+0000","last_clean":"2024-09-22T03:29:00.160849+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:29:00.160849+0000","last_undegraded":"2024-09-22T03:29:00.160849+0000","last_fullsized":"2024-09-22T03:29:00.160849+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"224'67","reported_seq":80,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:03.050147+0000","last_change":"2024-09-22T03:28:59.268256+0000","last_active":"2024-09-22T03:29:03.050147+0000","last_peered":"2024-09-22T03:29:03.050147+0000","last_clean":"2024-09-22T03:29:03.050147+0000","last_became_active":"2024-09-22T03:28:59.263256+0000","last_became_peered":"2024-09-22T03:28:59.263256+0000","last_unstale":"2024-09-22T03:29:03.050147+0000","last_undegraded":"2024-09-22T03:29:03.050147+0000","last_fullsized":"2024-09-22T03:29:03.050147+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":466,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.159043+0000","last_change":"2024-09-22T03:28:00.183512+0000","last_active":"2024-09-22T03:29:00.159043+0000","last_peered":"2024-09-22T03:29:00.159043+0000","last_clean":"2024-09-22T03:29:00.159043+0000","last_became_active":"2024-09-22T03:27:57.331679+0000","last_became_peered":"2024-09-22T03:27:57.331679+0000","last_unstale":"2024-09-22T03:29:00.159043+0000","last_undegraded":"2024-09-22T03:29:00.159043+0000","last_fullsized":"2024-09-22T03:29:00.159043+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"224'41","reported_seq":54,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:02.925495+0000","last_change":"2024-09-22T03:28:59.267581+0000","last_active":"2024-09-22T03:29:02.925495+0000","last_peered":"2024-09-22T03:29:02.925495+0000","last_clean":"2024-09-22T03:29:02.925495+0000","last_became_active":"2024-09-22T03:28:59.261143+0000","last_became_peered":"2024-09-22T03:28:59.261143+0000","last_unstale":"2024-09-22T03:29:02.925495+0000","last_undegraded":"2024-09-22T03:29:02.925495+0000","last_fullsized":"2024-09-22T03:29:02.925495+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":413,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.160625+0000","last_change":"2024-09-22T03:28:43.240328+0000","last_active":"2024-09-22T03:29:00.160625+0000","last_peered":"2024-09-22T03:29:00.160625+0000","last_clean":"2024-09-22T03:29:00.160625+0000","last_became_active":"2024-09-22T03:28:42.404462+0000","last_became_peered":"2024-09-22T03:28:42.404462+0000","last_unstale":"2024-09-22T03:29:00.160625+0000","last_undegraded":"2024-09-22T03:29:00.160625+0000","last_fullsized":"2024-09-22T03:29:00.160625+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"224'31","reported_seq":44,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.683093+0000","last_change":"2024-09-22T03:28:59.278638+0000","last_active":"2024-09-22T03:29:01.683093+0000","last_peered":"2024-09-22T03:29:01.683093+0000","last_clean":"2024-09-22T03:29:01.683093+0000","last_became_active":"2024-09-22T03:28:59.274902+0000","last_became_peered":"2024-09-22T03:28:59.274902+0000","last_unstale":"2024-09-22T03:29:01.683093+0000","last_undegraded":"2024-09-22T03:29:01.683093+0000","last_fullsized":"2024-09-22T03:29:01.683093+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":419,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.156659+0000","last_change":"2024-09-22T03:28:43.236213+0000","last_active":"2024-09-22T03:29:00.156659+0000","last_peered":"2024-09-22T03:29:00.156659+0000","last_clean":"2024-09-22T03:29:00.156659+0000","last_became_active":"2024-09-22T03:28:42.376613+0000","last_became_peered":"2024-09-22T03:28:42.376613+0000","last_unstale":"2024-09-22T03:29:00.156659+0000","last_undegraded":"2024-09-22T03:29:00.156659+0000","last_fullsized":"2024-09-22T03:29:00.156659+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":50921472,"num_objects":777,"num_object_clones":0,"num_object_copies":2331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":49728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8609792,"data_stored":137756672,"data_compressed":6538992,"data_compressed_allocated":8609792,"data_compressed_original":137756672,"omap_allocated":0,"internal_metadata":0},"log_size":761,"ondisk_log_size":761,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":104,"seq":446676599312,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95128,"kb_used_data":4616,"kb_used_omap":0,"kb_used_meta":90496,"kb_avail":94276712,"statfs":{"total":96636764160,"available":96539353088,"internally_reserved":0,"allocated":4726784,"data_stored":46646912,"data_compressed":2177595,"data_compressed_allocated":2867200,"data_compressed_original":45875200,"omap_allocated":0,"internal_metadata":92667904},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63440,"kb_used_data":3452,"kb_used_omap":0,"kb_used_meta":59968,"kb_avail":94308400,"statfs":{"total":96636764160,"available":96571801600,"internally_reserved":0,"allocated":3534848,"data_stored":27183264,"data_compressed":1255649,"data_compressed_allocated":1654784,"data_compressed_original":26394624,"omap_allocated":0,"internal_metadata":61407232},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40940,"kb_used_data":2972,"kb_used_omap":0,"kb_used_meta":37952,"kb_avail":94330900,"statfs":{"total":96636764160,"available":96594841600,"internally_reserved":0,"allocated":3043328,"data_stored":19318944,"data_compressed":882298,"data_compressed_allocated":1163264,"data_compressed_original":18530304,"omap_allocated":0,"internal_metadata":38862848},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328449,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49232,"kb_used_data":2752,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94322608,"statfs":{"total":96636764160,"available":96586350592,"internally_reserved":0,"allocated":2818048,"data_stored":15779981,"data_compressed":714312,"data_compressed_allocated":942080,"data_compressed_original":14991360,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1552384,"data_stored":24838144,"data_compressed":1178989,"data_compressed_allocated":1552384,"data_compressed_original":24838144,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1208320,"data_stored":19333120,"data_compressed":917657,"data_compressed_allocated":1208320,"data_compressed_original":19333120,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":33030144,"data_compressed":1567887,"data_compressed_allocated":2064384,"data_compressed_original":33030144,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3784704,"data_stored":60555264,"data_compressed":2874459,"data_compressed_allocated":3784704,"data_compressed_original":60555264,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:07.368 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:29:07.369 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:29:07.369 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [3, 2, 0, 1] 2024-09-22T03:29:07.369 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T03:29:07.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:29:08.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 8 16 2488 2472 19.3108 1.5 0.0483617 0.045622 2024-09-22T03:29:09.108 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:09.222+0000 7f51a7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:09.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:09.227+0000 7f51a7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:09.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 9 16 2488 2472 17.1652 0 - 0.045622 2024-09-22T03:29:10.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 10 16 2488 2472 15.4486 0 - 0.045622 2024-09-22T03:29:11.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T03:29:11.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 11 16 2488 2472 14.0442 0 - 0.045622 2024-09-22T03:29:12.296 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 12 16 2488 2472 12.8738 0 - 0.045622 2024-09-22T03:29:13.297 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 13 16 2488 2472 11.8835 0 - 0.045622 2024-09-22T03:29:13.471 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T03:29:14.297 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 14 16 2488 2472 11.0346 0 - 0.045622 2024-09-22T03:29:15.297 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 15 16 2488 2472 10.2989 0 - 0.045622 2024-09-22T03:29:16.297 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 16 16 2488 2472 9.6552 0 - 0.045622 2024-09-22T03:29:16.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T03:29:17.297 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 17 16 2488 2472 9.08723 0 - 0.045622 2024-09-22T03:29:18.297 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 18 16 2488 2472 8.58238 0 - 0.045622 2024-09-22T03:29:18.472 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [2, 0, 1] 2024-09-22T03:29:18.472 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:29:18.472 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-22T03:29:18.472 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:29:18.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:29:18.475 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:29:18.476 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:29:18.586 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:29:18.590 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:18.590 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:29:19.297 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 19 16 2488 2472 8.13066 0 - 0.045622 2024-09-22T03:29:20.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:29:20.297552+0000 min lat: 0.00875549 max lat: 0.101612 avg lat: 0.045622 2024-09-22T03:29:20.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:29:20.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 20 16 2488 2472 7.72411 0 - 0.045622 2024-09-22T03:29:21.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 21 16 2488 2472 7.35628 0 - 0.045622 2024-09-22T03:29:22.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 22 16 2488 2472 7.02189 0 - 0.045622 2024-09-22T03:29:23.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 23 16 2488 2472 6.71658 0 - 0.045622 2024-09-22T03:29:23.591 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:29:23.688 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:29:23.691 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:23.691 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:29:24.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 24 16 2488 2472 6.43673 0 - 0.045622 2024-09-22T03:29:25.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 25 16 2488 2472 6.17926 0 - 0.045622 2024-09-22T03:29:26.298 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 26 16 2488 2472 5.94159 0 - 0.045622 2024-09-22T03:29:27.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 27 16 2488 2472 5.72153 0 - 0.045622 2024-09-22T03:29:28.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 28 16 2488 2472 5.51717 0 - 0.045622 2024-09-22T03:29:28.692 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:29:28.789 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:29:28.791 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:28.792 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:29:29.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 29 16 2488 2472 5.32692 0 - 0.045622 2024-09-22T03:29:30.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 30 16 2488 2472 5.14936 0 - 0.045622 2024-09-22T03:29:31.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 31 16 2488 2472 4.98326 0 - 0.045622 2024-09-22T03:29:32.166 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:29:32.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:29:32.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:29:32.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 32 16 2488 2472 4.82753 0 - 0.045622 2024-09-22T03:29:33.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 33 16 2508 2492 4.71912 0.05 0.0136861 0.190824 2024-09-22T03:29:33.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:29:33.919 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:29:33.923 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:33.923 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:29:34.173 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:34.299 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 34 16 2988 2972 5.46257 30 0.0246 0.182899 2024-09-22T03:29:34.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:34.326+0000 7fcd4324f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:34.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:34.334+0000 7fcd4324f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:34.449 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:29:34.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:29:34.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:29:35.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 35 16 3435 3419 6.10462 27.9375 0.0140762 0.163676 2024-09-22T03:29:36.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 36 16 3840 3824 6.6381 25.3125 0.0691317 0.150142 2024-09-22T03:29:36.456 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:36.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:36.555+0000 7fce02157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:36.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:36.560+0000 7fce02157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:37.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 37 16 3840 3824 6.4587 0 - 0.150142 2024-09-22T03:29:38.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 38 16 4102 4086 6.71961 8.1875 0.0319521 0.148706 2024-09-22T03:29:38.627 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:38.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:38.768+0000 7fbdb5797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:38.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:38.774+0000 7fbdb5797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:38.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:29:39.129 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:29:39.129 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [ 2024-09-22T03:29:39.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.130 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.130 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 454, 2024-09-22T03:29:39.130 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.131 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.131 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.011896+0000" 2024-09-22T03:29:39.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.131 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.131 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.011907+0000" 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.132 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.012372+0000" 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.012681+0000" 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.012767+0000" 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.134 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.135 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.013023+0000" 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.014517+0000" 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.138 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.014640+0000" 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.015222+0000" 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.027948+0000" 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.143 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 457, 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.040213+0000" 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.040228+0000" 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.045136+0000" 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.147 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.048811+0000" 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.148 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.054407+0000" 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.058577+0000" 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.151 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.060125+0000" 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.064268+0000" 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.153 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.154 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.066415+0000" 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.073037+0000" 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.157 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.157 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.157 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.157 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.157 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.158 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.158 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.158 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.158 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery", 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 125, 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "pgid": "7.3", 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_detail": {} 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery_sub", 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 33554735, 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": {} 2024-09-22T03:29:39.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery_sub", 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 33554736, 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": {} 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 307, 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.161 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.311503+0000" 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.311509+0000" 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.162 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.312533+0000" 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.313907+0000" 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.314089+0000" 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.165 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.346607+0000" 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.314516+0000" 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.166 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.315634+0000" 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.167 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.168 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.315876+0000" 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.169 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.317471+0000" 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.172 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.320332+0000" 2024-09-22T03:29:39.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.174 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.174 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.174 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.175 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.346290+0000" 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.176 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 328, 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.410158+0000" 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.410165+0000" 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.177 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.410270+0000" 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.178 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.410331+0000" 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.180 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.410435+0000" 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.489614+0000" 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.182 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.411445+0000" 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.183 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.412384+0000" 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.412597+0000" 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.185 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.412751+0000" 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.418540+0000" 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.189 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.488282+0000" 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 349, 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.532970+0000" 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.192 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.532977+0000" 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.534047+0000" 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.193 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.534284+0000" 2024-09-22T03:29:39.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.535578+0000" 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.195 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.557467+0000" 2024-09-22T03:29:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.536687+0000" 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.197 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.538036+0000" 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.198 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.538550+0000" 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.201 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.539095+0000" 2024-09-22T03:29:39.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.203 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.543366+0000" 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.555675+0000" 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 356, 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.569612+0000" 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.206 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.569618+0000" 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.569699+0000" 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.569971+0000" 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.570071+0000" 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.601713+0000" 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.571539+0000" 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.571937+0000" 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.574475+0000" 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.574869+0000" 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.579785+0000" 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.597736+0000" 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 359, 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.579192+0000" 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.579200+0000" 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.580236+0000" 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.582444+0000" 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.584345+0000" 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.609038+0000" 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.584619+0000" 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.586091+0000" 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.586305+0000" 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.586970+0000" 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.593908+0000" 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.606945+0000" 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 366, 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.612744+0000" 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.612751+0000" 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.613524+0000" 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.613596+0000" 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.613695+0000" 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.643429+0000" 2024-09-22T03:29:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.614074+0000" 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.614279+0000" 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.615205+0000" 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.615624+0000" 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.624743+0000" 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.642145+0000" 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 372, 2024-09-22T03:29:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.636597+0000" 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.636603+0000" 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.637110+0000" 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.637373+0000" 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.638328+0000" 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.665904+0000" 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.638707+0000" 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.639122+0000" 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.639303+0000" 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.639548+0000" 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.644137+0000" 2024-09-22T03:29:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.663382+0000" 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 375, 2024-09-22T03:29:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.658224+0000" 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.658232+0000" 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.659283+0000" 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.659360+0000" 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.659563+0000" 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.742897+0000" 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.660378+0000" 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.661227+0000" 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.663370+0000" 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.665157+0000" 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.670161+0000" 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.741378+0000" 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 396, 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744721+0000" 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744725+0000" 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744793+0000" 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744858+0000" 2024-09-22T03:29:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744922+0000" 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.771165+0000" 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.745155+0000" 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.745404+0000" 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.745886+0000" 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.746142+0000" 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.749391+0000" 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.769746+0000" 2024-09-22T03:29:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 390, 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.714764+0000" 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.714775+0000" 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.716564+0000" 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.718495+0000" 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.720342+0000" 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.803806+0000" 2024-09-22T03:29:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.721799+0000" 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.722164+0000" 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.723788+0000" 2024-09-22T03:29:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.725557+0000" 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.731897+0000" 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.799576+0000" 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 406, 2024-09-22T03:29:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.784670+0000" 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.784683+0000" 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.785774+0000" 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.785956+0000" 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.786069+0000" 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.826038+0000" 2024-09-22T03:29:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.786471+0000" 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.786643+0000" 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.787967+0000" 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.788337+0000" 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.794861+0000" 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.824763+0000" 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 411, 2024-09-22T03:29:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.813183+0000" 2024-09-22T03:29:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.813191+0000" 2024-09-22T03:29:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.813349+0000" 2024-09-22T03:29:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.813464+0000" 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.813567+0000" 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.863162+0000" 2024-09-22T03:29:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.814787+0000" 2024-09-22T03:29:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.814941+0000" 2024-09-22T03:29:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.815109+0000" 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.816609+0000" 2024-09-22T03:29:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.818828+0000" 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.861000+0000" 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 414, 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.821567+0000" 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.821575+0000" 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.822393+0000" 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.822568+0000" 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.822666+0000" 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.877745+0000" 2024-09-22T03:29:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.823689+0000" 2024-09-22T03:29:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.823941+0000" 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.824083+0000" 2024-09-22T03:29:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.824667+0000" 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.827116+0000" 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.875984+0000" 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 427, 2024-09-22T03:29:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.876158+0000" 2024-09-22T03:29:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.876169+0000" 2024-09-22T03:29:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.876478+0000" 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.876600+0000" 2024-09-22T03:29:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.878281+0000" 2024-09-22T03:29:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.913894+0000" 2024-09-22T03:29:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.878825+0000" 2024-09-22T03:29:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.880256+0000" 2024-09-22T03:29:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.883182+0000" 2024-09-22T03:29:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.884894+0000" 2024-09-22T03:29:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.890361+0000" 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.911484+0000" 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 420, 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.841827+0000" 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.841845+0000" 2024-09-22T03:29:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.843529+0000" 2024-09-22T03:29:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.843583+0000" 2024-09-22T03:29:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.843694+0000" 2024-09-22T03:29:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.918781+0000" 2024-09-22T03:29:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.844039+0000" 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.845472+0000" 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.845684+0000" 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.845984+0000" 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.855146+0000" 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.402 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.402 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.402 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.918300+0000" 2024-09-22T03:29:39.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.404 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 425, 2024-09-22T03:29:39.404 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.404 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.871529+0000" 2024-09-22T03:29:39.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.405 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.405 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.871534+0000" 2024-09-22T03:29:39.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.406 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.406 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.406 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.406 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.871623+0000" 2024-09-22T03:29:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.409 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.409 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.872076+0000" 2024-09-22T03:29:39.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.409 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.872156+0000" 2024-09-22T03:29:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.410 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.411 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.412 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.951298+0000" 2024-09-22T03:29:39.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.412 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.413 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.872389+0000" 2024-09-22T03:29:39.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.413 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.415 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.415 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.872700+0000" 2024-09-22T03:29:39.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.416 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.416 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.416 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.416 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.416 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.873141+0000" 2024-09-22T03:29:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.419 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.420 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.420 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.420 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.420 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.421 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.421 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.422 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.422 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.422 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.873428+0000" 2024-09-22T03:29:39.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.422 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.423 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.423 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.424 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.424 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.424 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.878207+0000" 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.950209+0000" 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 437, 2024-09-22T03:29:39.427 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.930161+0000" 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.930172+0000" 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.428 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.930329+0000" 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.429 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.930427+0000" 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.930539+0000" 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.954868+0000" 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.930866+0000" 2024-09-22T03:29:39.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.933360+0000" 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.933634+0000" 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.435 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.436 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.935144+0000" 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.438 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.941228+0000" 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.439 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.952184+0000" 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.440 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 440, 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.956646+0000" 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.956651+0000" 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.443 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.443 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.443 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.956730+0000" 2024-09-22T03:29:39.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.443 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.956795+0000" 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.956886+0000" 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.445 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.009072+0000" 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.957290+0000" 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.957644+0000" 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.448 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.957886+0000" 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.449 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.450 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.450 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.958750+0000" 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.965417+0000" 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.006881+0000" 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 453, 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.008541+0000" 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.008551+0000" 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.008897+0000" 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.009316+0000" 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.009544+0000" 2024-09-22T03:29:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.062752+0000" 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.010029+0000" 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.010243+0000" 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.011135+0000" 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.011444+0000" 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.014152+0000" 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.056290+0000" 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 461, 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.081272+0000" 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.081287+0000" 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.081457+0000" 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.081546+0000" 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.081792+0000" 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.082040+0000" 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.082173+0000" 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.082307+0000" 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.082478+0000" 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.086218+0000" 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery", 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 16777337, 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "pgid": "7.2", 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_detail": {} 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery_sub", 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 33554737, 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": {} 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 392, 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.735476+0000" 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.735483+0000" 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.735738+0000" 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.737942+0000" 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.740585+0000" 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.775748+0000" 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.740865+0000" 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.740983+0000" 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.741097+0000" 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.741681+0000" 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.742321+0000" 2024-09-22T03:29:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.768854+0000" 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 393, 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.742808+0000" 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.742813+0000" 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.742991+0000" 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.743099+0000" 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.745472+0000" 2024-09-22T03:29:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.795241+0000" 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.746039+0000" 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.746278+0000" 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.746960+0000" 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.749014+0000" 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.753755+0000" 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.794741+0000" 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 398, 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.748245+0000" 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.748251+0000" 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.748324+0000" 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.748444+0000" 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.749288+0000" 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.797013+0000" 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.750502+0000" 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.750912+0000" 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.751702+0000" 2024-09-22T03:29:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.752129+0000" 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.759496+0000" 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.796470+0000" 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 395, 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744031+0000" 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744035+0000" 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744150+0000" 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.744296+0000" 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.745701+0000" 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.798439+0000" 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.746793+0000" 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.748525+0000" 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.749192+0000" 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.750676+0000" 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.757293+0000" 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.797942+0000" 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 399, 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.753693+0000" 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.753702+0000" 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.753779+0000" 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.756124+0000" 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.760170+0000" 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.807408+0000" 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.763501+0000" 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.765827+0000" 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.767433+0000" 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.769280+0000" 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.777795+0000" 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.806914+0000" 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 404, 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.772558+0000" 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.772568+0000" 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.773162+0000" 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.774762+0000" 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.776556+0000" 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.817937+0000" 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.777093+0000" 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.777340+0000" 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.778075+0000" 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.778517+0000" 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.785015+0000" 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.817425+0000" 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 403, 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.761633+0000" 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.761640+0000" 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.761717+0000" 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.761776+0000" 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.763921+0000" 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.818405+0000" 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.766688+0000" 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.767124+0000" 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.771889+0000" 2024-09-22T03:29:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.775000+0000" 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.782408+0000" 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.817439+0000" 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 409, 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.799367+0000" 2024-09-22T03:29:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.799373+0000" 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.803424+0000" 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.803755+0000" 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.805484+0000" 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.838634+0000" 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.805735+0000" 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.805863+0000" 2024-09-22T03:29:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.805974+0000" 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.806149+0000" 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.808081+0000" 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.838158+0000" 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 421, 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.867113+0000" 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.867124+0000" 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.867410+0000" 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.868658+0000" 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.868866+0000" 2024-09-22T03:29:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.887835+0000" 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.869119+0000" 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.869245+0000" 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.869354+0000" 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.869503+0000" 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.870494+0000" 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.887387+0000" 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 415, 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.824846+0000" 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.824854+0000" 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.825286+0000" 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.825691+0000" 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.826414+0000" 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.893381+0000" 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.826798+0000" 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.827086+0000" 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.827313+0000" 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.827603+0000" 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.829661+0000" 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.892207+0000" 2024-09-22T03:29:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 429, 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.891642+0000" 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.891649+0000" 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.895815+0000" 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.897270+0000" 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.897590+0000" 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.913084+0000" 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.897994+0000" 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.898183+0000" 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.898356+0000" 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.898628+0000" 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.899876+0000" 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.912613+0000" 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 423, 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.870416+0000" 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.870426+0000" 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.870527+0000" 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.870675+0000" 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.873130+0000" 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.931228+0000" 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.873620+0000" 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.873745+0000" 2024-09-22T03:29:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.873863+0000" 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.874049+0000" 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.876639+0000" 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.929671+0000" 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 432, 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.911583+0000" 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.911590+0000" 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.913705+0000" 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.914853+0000" 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.916800+0000" 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.943664+0000" 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.917194+0000" 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.917317+0000" 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.917535+0000" 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.918586+0000" 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.919871+0000" 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.942182+0000" 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 435, 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.922189+0000" 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.922195+0000" 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.922285+0000" 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.922372+0000" 2024-09-22T03:29:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.923955+0000" 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.943997+0000" 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.924481+0000" 2024-09-22T03:29:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.924777+0000" 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.924964+0000" 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.925214+0000" 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.926659+0000" 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.942295+0000" 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 439, 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.955906+0000" 2024-09-22T03:29:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.955912+0000" 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.955987+0000" 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.956204+0000" 2024-09-22T03:29:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.957750+0000" 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.989408+0000" 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.958168+0000" 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.958380+0000" 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.958859+0000" 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.959138+0000" 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.962054+0000" 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.989107+0000" 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 443, 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.963519+0000" 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.963530+0000" 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.964680+0000" 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.964795+0000" 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.967211+0000" 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.998288+0000" 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.968245+0000" 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.969620+0000" 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.969846+0000" 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.970201+0000" 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.973076+0000" 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.995945+0000" 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 450, 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.000495+0000" 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.000506+0000" 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.000832+0000" 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.000971+0000" 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.017215+0000" 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.054660+0000" 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.689 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.017745+0000" 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.690 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.019316+0000" 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.019789+0000" 2024-09-22T03:29:39.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.693 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.694 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.021567+0000" 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.695 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.026597+0000" 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.696 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.054372+0000" 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 455, 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.022396+0000" 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.022417+0000" 2024-09-22T03:29:39.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.023255+0000" 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.699 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.023437+0000" 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.700 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.025729+0000" 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.062158+0000" 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.026994+0000" 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.027245+0000" 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.703 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.027731+0000" 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.029898+0000" 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.040969+0000" 2024-09-22T03:29:39.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.708 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.061697+0000" 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 459, 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.068339+0000" 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.068350+0000" 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.068471+0000" 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.711 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.068550+0000" 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.712 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.070318+0000" 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.713 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.088597+0000" 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.070926+0000" 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.714 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.071142+0000" 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.715 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.071374+0000" 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.716 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.718 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.071621+0000" 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.719 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.072826+0000" 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.088270+0000" 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.721 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "replicated_request", 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 462, 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "RepRequest": { 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "reqid": "client.5653.0:4295", 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "pgid": "7.4", 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 228, 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "min_epoch": 227, 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "oid": "7:229a6d93:::benchmark_data_smithi146_54403_object4294:head", 2024-09-22T03:29:39.722 INFO:teuthology.orchestra.run.smithi148.stdout: "from": "2" 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "background_recovery_sub", 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 16777512, 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": {} 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 266, 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.723 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.137170+0000" 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.137181+0000" 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.137422+0000" 2024-09-22T03:29:39.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.725 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.137542+0000" 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.137769+0000" 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.168945+0000" 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.727 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.138258+0000" 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.139700+0000" 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.729 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.140158+0000" 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.730 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.731 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.140737+0000" 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.146328+0000" 2024-09-22T03:29:39.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.734 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.167622+0000" 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.735 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 286, 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.229650+0000" 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.229661+0000" 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.736 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.229829+0000" 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.737 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.231198+0000" 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.234697+0000" 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.261430+0000" 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.234957+0000" 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.740 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.235117+0000" 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.741 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.235228+0000" 2024-09-22T03:29:39.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.743 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.744 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.235387+0000" 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.236981+0000" 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.260017+0000" 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 293, 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.248574+0000" 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.748 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.248585+0000" 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.248709+0000" 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.749 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.248812+0000" 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.750 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.250628+0000" 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.270935+0000" 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.251004+0000" 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.251208+0000" 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.753 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.251403+0000" 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.755 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.251702+0000" 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.756 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.253638+0000" 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.757 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.270481+0000" 2024-09-22T03:29:39.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 305, 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.297581+0000" 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.297587+0000" 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.760 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.299869+0000" 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.300868+0000" 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.302137+0000" 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.322765+0000" 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.763 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.302440+0000" 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.764 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.302658+0000" 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.765 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.302770+0000" 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.766 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.302930+0000" 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.768 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.304855+0000" 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.769 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.322501+0000" 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.770 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 334, 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.458676+0000" 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.458689+0000" 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.458835+0000" 2024-09-22T03:29:39.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.773 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.458935+0000" 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.461873+0000" 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.493714+0000" 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.775 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.462111+0000" 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.776 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.462231+0000" 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.777 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.462341+0000" 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.778 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.462533+0000" 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.780 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.464867+0000" 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.781 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.782 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.492758+0000" 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 343, 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.500822+0000" 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.500832+0000" 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.784 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.500965+0000" 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.501089+0000" 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.502486+0000" 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.786 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.527439+0000" 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.787 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.502728+0000" 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.788 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.502832+0000" 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.502937+0000" 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.791 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.503105+0000" 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.792 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.793 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.503961+0000" 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.527153+0000" 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.795 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 345, 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.506351+0000" 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.506363+0000" 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.508572+0000" 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.797 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.509988+0000" 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.798 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.511406+0000" 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.799 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.530963+0000" 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.512555+0000" 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.512694+0000" 2024-09-22T03:29:39.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.802 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.512798+0000" 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.804 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.512974+0000" 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.515446+0000" 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.806 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.530216+0000" 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.807 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 363, 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605374+0000" 2024-09-22T03:29:39.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605379+0000" 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605480+0000" 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.809 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605551+0000" 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605830+0000" 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.811 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.623491+0000" 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.606471+0000" 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.608079+0000" 2024-09-22T03:29:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.608287+0000" 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.816 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.608537+0000" 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.817 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.610714+0000" 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.623131+0000" 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.819 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 364, 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.820 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605783+0000" 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605787+0000" 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.605905+0000" 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.606151+0000" 2024-09-22T03:29:39.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.609014+0000" 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.823 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.629603+0000" 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.609365+0000" 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.825 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.609564+0000" 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.609686+0000" 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.828 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.609911+0000" 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.829 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.615591+0000" 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.629334+0000" 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 374, 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.643929+0000" 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.643940+0000" 2024-09-22T03:29:39.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.645134+0000" 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.833 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.646291+0000" 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.649837+0000" 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.835 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.673050+0000" 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.651331+0000" 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.836 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.655373+0000" 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.837 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.655508+0000" 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.839 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.840 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.655691+0000" 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.841 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.657259+0000" 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.842 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.672753+0000" 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 379, 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.677856+0000" 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.677862+0000" 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.678068+0000" 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.846 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.678134+0000" 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.678506+0000" 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.848 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.708808+0000" 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.678763+0000" 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.679002+0000" 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.850 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.680103+0000" 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.851 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.680370+0000" 2024-09-22T03:29:39.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.682197+0000" 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.855 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.708458+0000" 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 407, 2024-09-22T03:29:39.856 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.791607+0000" 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.791614+0000" 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.857 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.791696+0000" 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.792537+0000" 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.792868+0000" 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.859 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.833719+0000" 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.860 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.793095+0000" 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.861 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.793214+0000" 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.862 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.793311+0000" 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.863 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.864 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.865 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.793462+0000" 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.795220+0000" 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.867 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.832868+0000" 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 413, 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.818201+0000" 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.818207+0000" 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.818405+0000" 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.870 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.871 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.818550+0000" 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.821232+0000" 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.846547+0000" 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.873 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.821539+0000" 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.821710+0000" 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.821847+0000" 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.822018+0000" 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.823231+0000" 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.845988+0000" 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 418, 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.833029+0000" 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.833040+0000" 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.833299+0000" 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.834960+0000" 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.835495+0000" 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.860120+0000" 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.835803+0000" 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.835998+0000" 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.836218+0000" 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.836412+0000" 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.837604+0000" 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.859467+0000" 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 431, 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.903216+0000" 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.903231+0000" 2024-09-22T03:29:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.904311+0000" 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.905610+0000" 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.909038+0000" 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.939323+0000" 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.909305+0000" 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.909441+0000" 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.909551+0000" 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.909707+0000" 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.911439+0000" 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.939021+0000" 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 433, 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.919029+0000" 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.919035+0000" 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.919154+0000" 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.919244+0000" 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.919732+0000" 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.945393+0000" 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.920022+0000" 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.920215+0000" 2024-09-22T03:29:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.920384+0000" 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.920661+0000" 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.921824+0000" 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.945114+0000" 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 442, 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.958250+0000" 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.958258+0000" 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.958864+0000" 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.959039+0000" 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.963244+0000" 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.993223+0000" 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.963907+0000" 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.966160+0000" 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.966666+0000" 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.966980+0000" 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.970435+0000" 2024-09-22T03:29:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.991988+0000" 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 446, 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.976828+0000" 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.976851+0000" 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.977111+0000" 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.978625+0000" 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.985232+0000" 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.024603+0000" 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.985784+0000" 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.987759+0000" 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.987974+0000" 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.988313+0000" 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:38.989152+0000" 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.024057+0000" 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client_request", 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 452, 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "detail": { 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.005330+0000" 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.005334+0000" 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.006126+0000" 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.006247+0000" 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.006823+0000" 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.031544+0000" 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.007208+0000" 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:29:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.007389+0000" 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.007566+0000" 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.007855+0000" 2024-09-22T03:29:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.009191+0000" 2024-09-22T03:29:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "blocking_event": { 2024-09-22T03:29:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-22T03:29:39.031271+0000" 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time_event": { 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "0.000000" 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-22T03:29:39.954 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:29:39.955 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 39 16 4373 4357 6.98156 16.9375 0.0642516 0.143082 2024-09-22T03:29:40.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:29:40.299874+0000 min lat: 0.00823066 max lat: 25.9505 avg lat: 0.139392 2024-09-22T03:29:40.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:29:40.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 40 16 4601 4585 7.16324 14.25 0.125723 0.139392 2024-09-22T03:29:41.071 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:41.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:41.208+0000 7f4d1df57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:41.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:41.215+0000 7f4d1df57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:41.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 41 16 4857 4841 7.37873 16 0.0916784 0.13544 2024-09-22T03:29:42.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 42 16 5128 5112 7.60627 16.9375 0.0529659 0.1314 2024-09-22T03:29:43.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 43 16 5468 5452 7.92352 21.25 0.0636523 0.126146 2024-09-22T03:29:43.485 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:43.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:43.630+0000 7f74e842c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:43.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:43.635+0000 7f74e842c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:44.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 44 16 5793 5777 8.20504 20.3125 0.0908269 0.121778 2024-09-22T03:29:44.957 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 0, 1, 3] 2024-09-22T03:29:44.957 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:29:44.957 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:45.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:45.104+0000 7f0cc0957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:45.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:45.114+0000 7f0cc0957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:45.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 45 16 6120 6104 8.47683 20.4375 0.0506526 0.117886 2024-09-22T03:29:45.405 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:29:45.406 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:29:45.422 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":781,"stamp":"2024-09-22T03:29:43.462201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":332530227,"num_objects":5069,"num_object_clones":0,"num_object_copies":15207,"num_objects_missing_on_primary":260,"num_objects_missing":260,"num_objects_degraded":658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5204,"num_write_kb":326056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":24576000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5151,"ondisk_log_size":5151,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":237228,"kb_used_data":63404,"kb_used_omap":0,"kb_used_meta":173760,"kb_avail":377250132,"statfs":{"total":386547056640,"available":386304135168,"internally_reserved":0,"allocated":64925696,"data_stored":919731933,"data_compressed":43526994,"data_compressed_allocated":57298944,"data_compressed_original":916537344,"omap_allocated":0,"internal_metadata":177930240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199753728,"num_objects":3048,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":260,"num_objects_missing":260,"num_objects_degraded":658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3048,"num_write_kb":195072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":24576000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001525"},"pg_stats":[{"pgid":"7.f","version":"228'298","reported_seq":246,"reported_epoch":228,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:29:41.406812+0000","last_change":"2024-09-22T03:29:37.307429+0000","last_active":"2024-09-22T03:29:41.406812+0000","last_peered":"2024-09-22T03:29:41.406812+0000","last_clean":"2024-09-22T03:29:07.259267+0000","last_became_active":"2024-09-22T03:29:37.302297+0000","last_became_peered":"2024-09-22T03:29:37.302297+0000","last_unstale":"2024-09-22T03:29:41.406812+0000","last_undegraded":"2024-09-22T03:29:37.203690+0000","last_fullsized":"2024-09-22T03:29:41.406812+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":299,"num_object_clones":0,"num_object_copies":897,"num_objects_missing_on_primary":75,"num_objects_missing":75,"num_objects_degraded":75,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":19136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":75},{"shards":"0,1,3","objects":224}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"228'289","reported_seq":314,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.281289+0000","last_change":"2024-09-22T03:28:59.274720+0000","last_active":"2024-09-22T03:29:41.281289+0000","last_peered":"2024-09-22T03:29:41.281289+0000","last_clean":"2024-09-22T03:29:41.281289+0000","last_became_active":"2024-09-22T03:28:59.242935+0000","last_became_peered":"2024-09-22T03:28:59.242935+0000","last_unstale":"2024-09-22T03:29:41.281289+0000","last_undegraded":"2024-09-22T03:29:41.281289+0000","last_fullsized":"2024-09-22T03:29:41.281289+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"228'343","reported_seq":379,"reported_epoch":228,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:29:42.831994+0000","last_change":"2024-09-22T03:29:37.419851+0000","last_active":"2024-09-22T03:29:42.831994+0000","last_peered":"2024-09-22T03:29:42.831994+0000","last_clean":"2024-09-22T03:29:07.355897+0000","last_became_active":"2024-09-22T03:29:37.299572+0000","last_became_peered":"2024-09-22T03:29:37.299572+0000","last_unstale":"2024-09-22T03:29:42.831994+0000","last_undegraded":"2024-09-22T03:29:37.204681+0000","last_fullsized":"2024-09-22T03:29:42.831994+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":22016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":84},{"shards":"1,2,3","objects":260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"228'315","reported_seq":426,"reported_epoch":228,"state":"active+recovering+degraded","last_fresh":"2024-09-22T03:29:42.980938+0000","last_change":"2024-09-22T03:29:39.769192+0000","last_active":"2024-09-22T03:29:42.980938+0000","last_peered":"2024-09-22T03:29:42.980938+0000","last_clean":"2024-09-22T03:29:07.323574+0000","last_became_active":"2024-09-22T03:29:37.364635+0000","last_became_peered":"2024-09-22T03:29:37.364635+0000","last_unstale":"2024-09-22T03:29:42.980938+0000","last_undegraded":"2024-09-22T03:29:37.236497+0000","last_fullsized":"2024-09-22T03:29:42.980938+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709376,"num_objects":316,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":316,"num_write_kb":20224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":2621440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":34},{"shards":"1,2,3","objects":282}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"228'297","reported_seq":366,"reported_epoch":228,"state":"active+recovering+degraded","last_fresh":"2024-09-22T03:29:41.532796+0000","last_change":"2024-09-22T03:29:39.560227+0000","last_active":"2024-09-22T03:29:41.532796+0000","last_peered":"2024-09-22T03:29:41.532796+0000","last_clean":"2024-09-22T03:29:07.287351+0000","last_became_active":"2024-09-22T03:29:37.272304+0000","last_became_peered":"2024-09-22T03:29:37.272304+0000","last_unstale":"2024-09-22T03:29:41.532796+0000","last_undegraded":"2024-09-22T03:29:37.188971+0000","last_fullsized":"2024-09-22T03:29:41.532796+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19464192,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":19008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":18},{"shards":"0,1,3","objects":279}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"228'292","reported_seq":240,"reported_epoch":228,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:29:41.523618+0000","last_change":"2024-09-22T03:29:37.342636+0000","last_active":"2024-09-22T03:29:41.523618+0000","last_peered":"2024-09-22T03:29:41.523618+0000","last_clean":"2024-09-22T03:29:07.273438+0000","last_became_active":"2024-09-22T03:29:37.334944+0000","last_became_peered":"2024-09-22T03:29:37.334944+0000","last_unstale":"2024-09-22T03:29:41.523618+0000","last_undegraded":"2024-09-22T03:29:37.198673+0000","last_fullsized":"2024-09-22T03:29:41.523618+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19202048,"num_objects":293,"num_object_clones":0,"num_object_copies":879,"num_objects_missing_on_primary":75,"num_objects_missing":75,"num_objects_degraded":75,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":293,"num_write_kb":18752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":75},{"shards":"0,2,3","objects":218}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"228'345","reported_seq":446,"reported_epoch":228,"state":"active+recovering+degraded","last_fresh":"2024-09-22T03:29:42.979849+0000","last_change":"2024-09-22T03:29:40.226567+0000","last_active":"2024-09-22T03:29:42.979849+0000","last_peered":"2024-09-22T03:29:42.979849+0000","last_clean":"2024-09-22T03:29:07.371997+0000","last_became_active":"2024-09-22T03:29:37.369277+0000","last_became_peered":"2024-09-22T03:29:37.369277+0000","last_unstale":"2024-09-22T03:29:42.979849+0000","last_undegraded":"2024-09-22T03:29:37.218641+0000","last_fullsized":"2024-09-22T03:29:42.979849+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":54},{"shards":"1,2,3","objects":291}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"228'320","reported_seq":438,"reported_epoch":228,"state":"active+recovering+degraded","last_fresh":"2024-09-22T03:29:41.337635+0000","last_change":"2024-09-22T03:29:40.119164+0000","last_active":"2024-09-22T03:29:41.337635+0000","last_peered":"2024-09-22T03:29:41.337635+0000","last_clean":"2024-09-22T03:29:07.345184+0000","last_became_active":"2024-09-22T03:29:37.309210+0000","last_became_peered":"2024-09-22T03:29:37.309210+0000","last_unstale":"2024-09-22T03:29:41.337635+0000","last_undegraded":"2024-09-22T03:29:37.197113+0000","last_fullsized":"2024-09-22T03:29:41.337635+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037056,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":321,"num_write_kb":20544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":52},{"shards":"0,1,3","objects":269}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"228'348","reported_seq":280,"reported_epoch":228,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:29:41.448256+0000","last_change":"2024-09-22T03:29:37.340576+0000","last_active":"2024-09-22T03:29:41.448256+0000","last_peered":"2024-09-22T03:29:41.448256+0000","last_clean":"2024-09-22T03:29:07.350563+0000","last_became_active":"2024-09-22T03:29:37.314865+0000","last_became_peered":"2024-09-22T03:29:37.314865+0000","last_unstale":"2024-09-22T03:29:41.448256+0000","last_undegraded":"2024-09-22T03:29:37.194222+0000","last_fullsized":"2024-09-22T03:29:41.448256+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22872064,"num_objects":349,"num_object_clones":0,"num_object_copies":1047,"num_objects_missing_on_primary":92,"num_objects_missing":92,"num_objects_degraded":92,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":349,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":92},{"shards":"0,1,3","objects":257}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":400,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.342926+0000","last_change":"2024-09-22T03:29:37.343549+0000","last_active":"2024-09-22T03:29:37.342926+0000","last_peered":"2024-09-22T03:29:37.342926+0000","last_clean":"2024-09-22T03:27:20.147422+0000","last_became_active":"2024-09-22T03:29:37.333954+0000","last_became_peered":"2024-09-22T03:29:37.333954+0000","last_unstale":"2024-09-22T03:29:37.342926+0000","last_undegraded":"2024-09-22T03:29:37.342926+0000","last_fullsized":"2024-09-22T03:29:37.342926+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"228'318","reported_seq":531,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.336382+0000","last_change":"2024-09-22T03:29:40.136993+0000","last_active":"2024-09-22T03:29:41.336382+0000","last_peered":"2024-09-22T03:29:41.336382+0000","last_clean":"2024-09-22T03:29:41.336382+0000","last_became_active":"2024-09-22T03:29:37.304641+0000","last_became_peered":"2024-09-22T03:29:37.304641+0000","last_unstale":"2024-09-22T03:29:41.336382+0000","last_undegraded":"2024-09-22T03:29:41.336382+0000","last_fullsized":"2024-09-22T03:29:41.336382+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20905984,"num_objects":319,"num_object_clones":0,"num_object_copies":957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":319,"num_write_kb":20416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":5898240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":520,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.189320+0000","last_change":"2024-09-22T03:28:43.306144+0000","last_active":"2024-09-22T03:29:37.189320+0000","last_peered":"2024-09-22T03:29:37.189320+0000","last_clean":"2024-09-22T03:29:37.189320+0000","last_became_active":"2024-09-22T03:28:42.347727+0000","last_became_peered":"2024-09-22T03:28:42.347727+0000","last_unstale":"2024-09-22T03:29:37.189320+0000","last_undegraded":"2024-09-22T03:29:37.189320+0000","last_fullsized":"2024-09-22T03:29:37.189320+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"228'339","reported_seq":375,"reported_epoch":228,"state":"active+recovering+degraded","last_fresh":"2024-09-22T03:29:42.955284+0000","last_change":"2024-09-22T03:29:42.864506+0000","last_active":"2024-09-22T03:29:42.955284+0000","last_peered":"2024-09-22T03:29:42.955284+0000","last_clean":"2024-09-22T03:29:07.354859+0000","last_became_active":"2024-09-22T03:29:37.372303+0000","last_became_peered":"2024-09-22T03:29:37.372303+0000","last_unstale":"2024-09-22T03:29:42.955284+0000","last_undegraded":"2024-09-22T03:29:37.223634+0000","last_fullsized":"2024-09-22T03:29:42.955284+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":340,"num_object_clones":0,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":88},{"shards":"0,2,3","objects":252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":588,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.325507+0000","last_change":"2024-09-22T03:29:37.326433+0000","last_active":"2024-09-22T03:29:37.325507+0000","last_peered":"2024-09-22T03:29:37.325507+0000","last_clean":"2024-09-22T03:29:00.163818+0000","last_became_active":"2024-09-22T03:29:37.321358+0000","last_became_peered":"2024-09-22T03:29:37.321358+0000","last_unstale":"2024-09-22T03:29:37.325507+0000","last_undegraded":"2024-09-22T03:29:37.325507+0000","last_fullsized":"2024-09-22T03:29:37.325507+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"228'298","reported_seq":477,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.275106+0000","last_change":"2024-09-22T03:29:39.750757+0000","last_active":"2024-09-22T03:29:41.275106+0000","last_peered":"2024-09-22T03:29:41.275106+0000","last_clean":"2024-09-22T03:29:41.275106+0000","last_became_active":"2024-09-22T03:29:37.299085+0000","last_became_peered":"2024-09-22T03:29:37.299085+0000","last_unstale":"2024-09-22T03:29:41.275106+0000","last_undegraded":"2024-09-22T03:29:41.275106+0000","last_fullsized":"2024-09-22T03:29:41.275106+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":299,"num_object_clones":0,"num_object_copies":897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":19136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":4784128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":506,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.297944+0000","last_change":"2024-09-22T03:29:37.298796+0000","last_active":"2024-09-22T03:29:37.297944+0000","last_peered":"2024-09-22T03:29:37.297944+0000","last_clean":"2024-09-22T03:11:05.253234+0000","last_became_active":"2024-09-22T03:29:37.276538+0000","last_became_peered":"2024-09-22T03:29:37.276538+0000","last_unstale":"2024-09-22T03:29:37.297944+0000","last_undegraded":"2024-09-22T03:29:37.297944+0000","last_fullsized":"2024-09-22T03:29:37.297944+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":637,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.192083+0000","last_change":"2024-09-22T03:28:42.474401+0000","last_active":"2024-09-22T03:29:37.192083+0000","last_peered":"2024-09-22T03:29:37.192083+0000","last_clean":"2024-09-22T03:29:37.192083+0000","last_became_active":"2024-09-22T03:28:42.463122+0000","last_became_peered":"2024-09-22T03:28:42.463122+0000","last_unstale":"2024-09-22T03:29:37.192083+0000","last_undegraded":"2024-09-22T03:29:37.192083+0000","last_fullsized":"2024-09-22T03:29:37.192083+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"228'301","reported_seq":400,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.342546+0000","last_change":"2024-09-22T03:29:39.545514+0000","last_active":"2024-09-22T03:29:41.342546+0000","last_peered":"2024-09-22T03:29:41.342546+0000","last_clean":"2024-09-22T03:29:41.342546+0000","last_became_active":"2024-09-22T03:29:37.290473+0000","last_became_peered":"2024-09-22T03:29:37.290473+0000","last_unstale":"2024-09-22T03:29:41.342546+0000","last_undegraded":"2024-09-22T03:29:41.342546+0000","last_fullsized":"2024-09-22T03:29:41.342546+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302,"num_write_kb":19328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":354,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.201377+0000","last_change":"2024-09-22T03:28:43.239969+0000","last_active":"2024-09-22T03:29:37.201377+0000","last_peered":"2024-09-22T03:29:37.201377+0000","last_clean":"2024-09-22T03:29:37.201377+0000","last_became_active":"2024-09-22T03:28:42.448247+0000","last_became_peered":"2024-09-22T03:28:42.448247+0000","last_unstale":"2024-09-22T03:29:37.201377+0000","last_undegraded":"2024-09-22T03:29:37.201377+0000","last_fullsized":"2024-09-22T03:29:37.201377+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"228'305","reported_seq":423,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.469443+0000","last_change":"2024-09-22T03:29:40.485433+0000","last_active":"2024-09-22T03:29:41.469443+0000","last_peered":"2024-09-22T03:29:41.469443+0000","last_clean":"2024-09-22T03:29:41.469443+0000","last_became_active":"2024-09-22T03:29:37.314425+0000","last_became_peered":"2024-09-22T03:29:37.314425+0000","last_unstale":"2024-09-22T03:29:41.469443+0000","last_undegraded":"2024-09-22T03:29:41.469443+0000","last_fullsized":"2024-09-22T03:29:41.469443+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":306,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":19584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":482,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.278628+0000","last_change":"2024-09-22T03:29:37.279264+0000","last_active":"2024-09-22T03:29:37.278628+0000","last_peered":"2024-09-22T03:29:37.278628+0000","last_clean":"2024-09-22T03:29:00.159943+0000","last_became_active":"2024-09-22T03:29:37.276360+0000","last_became_peered":"2024-09-22T03:29:37.276360+0000","last_unstale":"2024-09-22T03:29:37.278628+0000","last_undegraded":"2024-09-22T03:29:37.278628+0000","last_fullsized":"2024-09-22T03:29:37.278628+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"228'334","reported_seq":368,"reported_epoch":228,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-22T03:29:42.743754+0000","last_change":"2024-09-22T03:29:37.413353+0000","last_active":"2024-09-22T03:29:42.743754+0000","last_peered":"2024-09-22T03:29:42.743754+0000","last_clean":"2024-09-22T03:29:07.267402+0000","last_became_active":"2024-09-22T03:29:37.311970+0000","last_became_peered":"2024-09-22T03:29:37.311970+0000","last_unstale":"2024-09-22T03:29:42.743754+0000","last_undegraded":"2024-09-22T03:29:37.193174+0000","last_fullsized":"2024-09-22T03:29:42.743754+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21954560,"num_objects":335,"num_object_clones":0,"num_object_copies":1005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":335,"num_write_kb":21440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":86},{"shards":"1,2,3","objects":249}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":429,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.300770+0000","last_change":"2024-09-22T03:29:37.301258+0000","last_active":"2024-09-22T03:29:37.300770+0000","last_peered":"2024-09-22T03:29:37.300770+0000","last_clean":"2024-09-22T03:29:00.161106+0000","last_became_active":"2024-09-22T03:29:37.298540+0000","last_became_peered":"2024-09-22T03:29:37.298540+0000","last_unstale":"2024-09-22T03:29:37.300770+0000","last_undegraded":"2024-09-22T03:29:37.300770+0000","last_fullsized":"2024-09-22T03:29:37.300770+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"228'309","reported_seq":513,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.847740+0000","last_change":"2024-09-22T03:29:40.097140+0000","last_active":"2024-09-22T03:29:41.847740+0000","last_peered":"2024-09-22T03:29:41.847740+0000","last_clean":"2024-09-22T03:29:41.847740+0000","last_became_active":"2024-09-22T03:29:37.295754+0000","last_became_peered":"2024-09-22T03:29:37.295754+0000","last_unstale":"2024-09-22T03:29:41.847740+0000","last_undegraded":"2024-09-22T03:29:41.847740+0000","last_fullsized":"2024-09-22T03:29:41.847740+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20316160,"num_objects":310,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":19840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":5570560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":435,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-22T03:29:37.287618+0000","last_change":"2024-09-22T03:29:37.288137+0000","last_active":"2024-09-22T03:29:37.287618+0000","last_peered":"2024-09-22T03:29:37.287618+0000","last_clean":"2024-09-22T03:29:00.157098+0000","last_became_active":"2024-09-22T03:29:37.282395+0000","last_became_peered":"2024-09-22T03:29:37.282395+0000","last_unstale":"2024-09-22T03:29:37.287618+0000","last_undegraded":"2024-09-22T03:29:37.287618+0000","last_fullsized":"2024-09-22T03:29:37.287618+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":331939840,"num_objects":5065,"num_object_clones":0,"num_object_copies":15195,"num_objects_missing_on_primary":260,"num_objects_missing":260,"num_objects_degraded":658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5065,"num_write_kb":324160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":24576000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58683392,"data_stored":938934272,"data_compressed":44582294,"data_compressed_allocated":58683392,"data_compressed_original":938934272,"omap_allocated":0,"internal_metadata":0},"log_size":5051,"ondisk_log_size":5051,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":227,"seq":974957576197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63076,"kb_used_data":17044,"kb_used_omap":0,"kb_used_meta":46016,"kb_avail":94308764,"statfs":{"total":96636764160,"available":96572174336,"internally_reserved":0,"allocated":17453056,"data_stored":249757220,"data_compressed":11821438,"data_compressed_allocated":15560704,"data_compressed_original":248971264,"omap_allocated":0,"internal_metadata":47120384},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65060,"kb_used_data":13396,"kb_used_omap":0,"kb_used_meta":51648,"kb_avail":94306780,"statfs":{"total":96636764160,"available":96570142720,"internally_reserved":0,"allocated":13717504,"data_stored":189595716,"data_compressed":8967030,"data_compressed_allocated":11804672,"data_compressed_original":188792832,"omap_allocated":0,"internal_metadata":52887552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51512,"kb_used_data":17704,"kb_used_omap":0,"kb_used_meta":33792,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":18128896,"data_stored":260177988,"data_compressed":12318370,"data_compressed_allocated":16216064,"data_compressed_original":259375104,"omap_allocated":0,"internal_metadata":34603008},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57580,"kb_used_data":15260,"kb_used_omap":0,"kb_used_meta":42304,"kb_avail":94314260,"statfs":{"total":96636764160,"available":96577802240,"internally_reserved":0,"allocated":15626240,"data_stored":220201009,"data_compressed":10420156,"data_compressed_allocated":13717504,"data_compressed_original":219398144,"omap_allocated":0,"internal_metadata":43319296},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14147584,"data_stored":226361344,"data_compressed":10748064,"data_compressed_allocated":14147584,"data_compressed_original":226361344,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16228352,"data_stored":259653632,"data_compressed":12328854,"data_compressed_allocated":16228352,"data_compressed_original":259653632,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12218368,"data_stored":195493888,"data_compressed":9282490,"data_compressed_allocated":12218368,"data_compressed_original":195493888,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16089088,"data_stored":257425408,"data_compressed":12222886,"data_compressed_allocated":16089088,"data_compressed_original":257425408,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:45.903 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:46.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:46.042+0000 7f9786d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:46.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:46.050+0000 7f9786d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:46.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 46 16 6426 6410 8.70827 19.125 0.0369801 0.114748 2024-09-22T03:29:47.300 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 47 16 6769 6753 8.97906 21.4375 0.0814103 0.111292 2024-09-22T03:29:48.301 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 48 16 7112 7096 9.23856 21.4375 0.0464662 0.108157 2024-09-22T03:29:48.349 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:48.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:48.495+0000 7f4cee4b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:48.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:48.502+0000 7f4cee4b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:49.301 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 49 16 7456 7440 9.48875 21.5 0.0126637 0.105317 2024-09-22T03:29:50.301 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 50 16 7813 7797 9.74518 22.3125 0.0544611 0.10256 2024-09-22T03:29:50.424 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 0, 1, 3] 2024-09-22T03:29:50.424 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:29:50.425 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [2, 0, 1, 3] 2024-09-22T03:29:50.425 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T03:29:50.425 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:29:50.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:50.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:50.746+0000 7f5433357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:50.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:29:50.751+0000 7f5433357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:51.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-22T03:29:51.301 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 51 16 7876 7860 9.63128 3.9375 0.0144199 0.102062 2024-09-22T03:29:52.301 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 52 16 7876 7860 9.44606 0 - 0.102062 2024-09-22T03:29:53.301 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 53 16 7876 7860 9.26784 0 - 0.102062 2024-09-22T03:29:54.301 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 54 16 7876 7860 9.0962 0 - 0.102062 2024-09-22T03:29:55.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 55 16 7876 7860 8.93081 0 - 0.102062 2024-09-22T03:29:56.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 56 16 7876 7860 8.77132 0 - 0.102062 2024-09-22T03:29:56.526 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T03:29:56.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~6s 2024-09-22T03:29:57.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 57 16 7876 7860 8.61743 0 - 0.102062 2024-09-22T03:29:58.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 58 16 7876 7860 8.46886 0 - 0.102062 2024-09-22T03:29:59.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 59 16 7876 7860 8.32532 0 - 0.102062 2024-09-22T03:30:00.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:30:00.302178+0000 min lat: 0.00823066 max lat: 25.9505 avg lat: 0.102062 2024-09-22T03:30:00.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:30:00.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 60 16 7876 7860 8.18656 0 - 0.102062 2024-09-22T03:30:01.302 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 61 16 7876 7860 8.05235 0 - 0.102062 2024-09-22T03:30:01.527 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 1, 3] 2024-09-22T03:30:01.527 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:30:01.527 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [2, 1, 3] 2024-09-22T03:30:01.528 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T03:30:01.528 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:30:01.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T03:30:01.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-22T03:30:02.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 62 16 7876 7860 7.92247 0 - 0.102062 2024-09-22T03:30:03.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 63 16 7876 7860 7.79671 0 - 0.102062 2024-09-22T03:30:04.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 64 16 7876 7860 7.67489 0 - 0.102062 2024-09-22T03:30:05.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 65 16 7876 7860 7.55682 0 - 0.102062 2024-09-22T03:30:06.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 66 16 7876 7860 7.44232 0 - 0.102062 2024-09-22T03:30:07.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 67 16 7876 7860 7.33124 0 - 0.102062 2024-09-22T03:30:07.377 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~5s 2024-09-22T03:30:07.628 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T03:30:08.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 68 16 7876 7860 7.22343 0 - 0.102062 2024-09-22T03:30:09.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 69 16 7876 7860 7.11874 0 - 0.102062 2024-09-22T03:30:10.303 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 70 16 7876 7860 7.01704 0 - 0.102062 2024-09-22T03:30:11.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 71 16 7876 7860 6.91821 0 - 0.102062 2024-09-22T03:30:12.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 72 16 7876 7860 6.82212 0 - 0.102062 2024-09-22T03:30:12.629 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 3] live_osds: [2, 1] 2024-09-22T03:30:12.629 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:30:12.629 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-22T03:30:12.629 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:30:12.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:30:12.632 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:30:12.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:30:12.753 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:30:12.756 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:12.757 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:30:12.780 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T03:30:13.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 73 16 7876 7860 6.72866 0 - 0.102062 2024-09-22T03:30:14.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 74 16 7876 7860 6.63773 0 - 0.102062 2024-09-22T03:30:15.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 75 16 7876 7860 6.54923 0 - 0.102062 2024-09-22T03:30:16.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 76 16 7876 7860 6.46305 0 - 0.102062 2024-09-22T03:30:17.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 77 16 7876 7860 6.37912 0 - 0.102062 2024-09-22T03:30:17.757 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:30:17.854 INFO:teuthology.orchestra.run.smithi148.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:30:17.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:17.857 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-22T03:30:18.304 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 78 16 7876 7860 6.29733 0 - 0.102062 2024-09-22T03:30:19.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 79 16 7876 7860 6.21762 0 - 0.102062 2024-09-22T03:30:20.166 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:20.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:2024-09-22T03:30:20.304771+0000 min lat: 0.00823066 max lat: 25.9505 avg lat: 0.102062 2024-09-22T03:30:20.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:30:20.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 80 16 7876 7860 6.13989 0 - 0.102062 2024-09-22T03:30:21.178 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-22T03:30:21.183 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:21.184 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:21.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 81 16 7876 7860 6.06409 0 - 0.102062 2024-09-22T03:30:22.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 82 16 7876 7860 5.99014 0 - 0.102062 2024-09-22T03:30:22.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T03:30:22.965 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-22T03:30:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "ops_in_flight": [] 2024-09-22T03:30:22.965 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-22T03:30:23.185 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:23.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 83 16 7878 7862 5.91948 0.00390625 32.8429 0.110388 2024-09-22T03:30:23.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:23.325+0000 7fae3c517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:23.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:23.330+0000 7fae3c517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:23.335 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:23.486 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:23.487 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:24.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 84 16 8247 8231 6.12353 23.0625 0.0285546 0.163251 2024-09-22T03:30:25.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 85 16 8631 8615 6.33381 24 0.0737564 0.157839 2024-09-22T03:30:25.488 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:25.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:25.630+0000 7fa164b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:25.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:25.637+0000 7fa164b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:25.645 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:25.837 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:25.837 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:26.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 86 16 8989 8973 6.52031 22.375 0.030608 0.15332 2024-09-22T03:30:27.305 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 87 16 9360 9344 6.71186 23.1875 0.062376 0.148943 2024-09-22T03:30:27.838 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:27.975 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 1, 3] 2024-09-22T03:30:27.975 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:30:27.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:30:27.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:27.984+0000 7f55438fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:27.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:27.991+0000 7f55438fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:27.998 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:28.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:28.120+0000 7f89a5e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:28.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:28.127+0000 7f89a5e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:28.204 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:28.205 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:28.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 88 16 9731 9715 6.89905 23.1875 0.0314589 0.144905 2024-09-22T03:30:28.394 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:30:28.395 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:30:28.412 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":807,"stamp":"2024-09-22T03:30:27.471787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":570294835,"num_objects":8697,"num_object_clones":0,"num_object_copies":26091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8832,"num_write_kb":558248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1260,"num_bytes_recovered":50659328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8782,"ondisk_log_size":8782,"up":57,"acting":57,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":331704,"kb_used_data":105592,"kb_used_omap":0,"kb_used_meta":226048,"kb_avail":377155656,"statfs":{"total":386547056640,"available":386207391744,"internally_reserved":0,"allocated":108126208,"data_stored":1609538052,"data_compressed":76280794,"data_compressed_allocated":100409344,"data_compressed_original":1606303744,"omap_allocated":0,"internal_metadata":231473152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":58785792,"num_objects":897,"num_object_clones":0,"num_object_copies":2691,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":897,"num_write_kb":57408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.295983"},"pg_stats":[{"pgid":"7.f","version":"232'574","reported_seq":682,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.576613+0000","last_change":"2024-09-22T03:30:23.269358+0000","last_active":"2024-09-22T03:30:26.576613+0000","last_peered":"2024-09-22T03:30:26.576613+0000","last_clean":"2024-09-22T03:29:50.473527+0000","last_became_active":"2024-09-22T03:30:23.269358+0000","last_became_peered":"2024-09-22T03:30:23.269358+0000","last_unstale":"2024-09-22T03:30:26.576613+0000","last_undegraded":"2024-09-22T03:30:23.204838+0000","last_fullsized":"2024-09-22T03:30:23.203911+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":575,"num_object_clones":0,"num_object_copies":1725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":575,"num_write_kb":36800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":575}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"232'557","reported_seq":593,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.284739+0000","last_change":"2024-09-22T03:30:21.227133+0000","last_active":"2024-09-22T03:30:26.284739+0000","last_peered":"2024-09-22T03:30:26.284739+0000","last_clean":"2024-09-22T03:29:50.298165+0000","last_became_active":"2024-09-22T03:30:21.227133+0000","last_became_peered":"2024-09-22T03:30:21.227133+0000","last_unstale":"2024-09-22T03:30:26.284739+0000","last_undegraded":"2024-09-22T03:30:21.199511+0000","last_fullsized":"2024-09-22T03:30:21.198823+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36569088,"num_objects":558,"num_object_clones":0,"num_object_copies":1674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":558,"num_write_kb":35712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":558}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"228'490","reported_seq":701,"reported_epoch":231,"state":"peering","last_fresh":"2024-09-22T03:30:21.192445+0000","last_change":"2024-09-22T03:30:21.192445+0000","last_active":"2024-09-22T03:30:21.192445+0000","last_peered":"2024-09-22T03:30:20.173142+0000","last_clean":"2024-09-22T03:30:20.173142+0000","last_became_active":"2024-09-22T03:29:37.299572+0000","last_became_peered":"2024-09-22T03:29:37.299572+0000","last_unstale":"2024-09-22T03:30:21.192445+0000","last_undegraded":"2024-09-22T03:30:21.192445+0000","last_fullsized":"2024-09-22T03:30:21.192445+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32112640,"num_objects":490,"num_object_clones":0,"num_object_copies":1470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":490,"num_write_kb":31360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":5636096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"228'462","reported_seq":647,"reported_epoch":231,"state":"peering","last_fresh":"2024-09-22T03:30:21.201587+0000","last_change":"2024-09-22T03:30:21.201587+0000","last_active":"2024-09-22T03:30:21.201587+0000","last_peered":"2024-09-22T03:30:20.182532+0000","last_clean":"2024-09-22T03:30:20.182532+0000","last_became_active":"2024-09-22T03:29:37.364635+0000","last_became_peered":"2024-09-22T03:29:37.364635+0000","last_unstale":"2024-09-22T03:30:21.201587+0000","last_undegraded":"2024-09-22T03:30:21.201587+0000","last_fullsized":"2024-09-22T03:30:21.201587+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":462,"num_object_clones":0,"num_object_copies":1386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":462,"num_write_kb":29568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4849664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"232'582","reported_seq":697,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.529702+0000","last_change":"2024-09-22T03:30:23.256501+0000","last_active":"2024-09-22T03:30:26.529702+0000","last_peered":"2024-09-22T03:30:26.529702+0000","last_clean":"2024-09-22T03:29:50.461397+0000","last_became_active":"2024-09-22T03:30:23.256501+0000","last_became_peered":"2024-09-22T03:30:23.256501+0000","last_unstale":"2024-09-22T03:30:26.529702+0000","last_undegraded":"2024-09-22T03:30:23.200922+0000","last_fullsized":"2024-09-22T03:30:23.200281+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38207488,"num_objects":583,"num_object_clones":0,"num_object_copies":1749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":37312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":583}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"232'549","reported_seq":657,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.440426+0000","last_change":"2024-09-22T03:30:23.247920+0000","last_active":"2024-09-22T03:30:26.440426+0000","last_peered":"2024-09-22T03:30:26.440426+0000","last_clean":"2024-09-22T03:29:50.455169+0000","last_became_active":"2024-09-22T03:30:23.247920+0000","last_became_peered":"2024-09-22T03:30:23.247920+0000","last_unstale":"2024-09-22T03:30:26.440426+0000","last_undegraded":"2024-09-22T03:30:23.201360+0000","last_fullsized":"2024-09-22T03:30:23.200407+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36044800,"num_objects":550,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":550,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":550,"num_write_kb":35200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":550}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"228'492","reported_seq":708,"reported_epoch":231,"state":"peering","last_fresh":"2024-09-22T03:30:21.199411+0000","last_change":"2024-09-22T03:30:21.199411+0000","last_active":"2024-09-22T03:30:21.199411+0000","last_peered":"2024-09-22T03:30:20.183144+0000","last_clean":"2024-09-22T03:30:20.183144+0000","last_became_active":"2024-09-22T03:29:37.369277+0000","last_became_peered":"2024-09-22T03:29:37.369277+0000","last_unstale":"2024-09-22T03:30:21.199411+0000","last_undegraded":"2024-09-22T03:30:21.199411+0000","last_fullsized":"2024-09-22T03:30:21.199411+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32243712,"num_objects":492,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":31488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":5898240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"232'569","reported_seq":802,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.331937+0000","last_change":"2024-09-22T03:30:23.252835+0000","last_active":"2024-09-22T03:30:26.331937+0000","last_peered":"2024-09-22T03:30:26.331937+0000","last_clean":"2024-09-22T03:29:50.441556+0000","last_became_active":"2024-09-22T03:30:23.252835+0000","last_became_peered":"2024-09-22T03:30:23.252835+0000","last_unstale":"2024-09-22T03:30:26.331937+0000","last_undegraded":"2024-09-22T03:30:23.202632+0000","last_fullsized":"2024-09-22T03:30:23.201941+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":570,"num_object_clones":0,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":570,"num_write_kb":36480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":6291456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":570}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"232'578","reported_seq":704,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.554475+0000","last_change":"2024-09-22T03:30:23.262900+0000","last_active":"2024-09-22T03:30:26.554475+0000","last_peered":"2024-09-22T03:30:26.554475+0000","last_clean":"2024-09-22T03:29:50.442823+0000","last_became_active":"2024-09-22T03:30:23.262900+0000","last_became_peered":"2024-09-22T03:30:23.262900+0000","last_unstale":"2024-09-22T03:30:26.554475+0000","last_undegraded":"2024-09-22T03:30:23.205025+0000","last_fullsized":"2024-09-22T03:30:23.204102+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37945344,"num_objects":579,"num_object_clones":0,"num_object_copies":1737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":579,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":579,"num_write_kb":37056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":579}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":407,"reported_epoch":232,"state":"active+undersized","last_fresh":"2024-09-22T03:30:23.245464+0000","last_change":"2024-09-22T03:30:23.245127+0000","last_active":"2024-09-22T03:30:23.245464+0000","last_peered":"2024-09-22T03:30:23.245464+0000","last_clean":"2024-09-22T03:29:37.343549+0000","last_became_active":"2024-09-22T03:30:23.245127+0000","last_became_peered":"2024-09-22T03:30:23.245127+0000","last_unstale":"2024-09-22T03:30:23.245464+0000","last_undegraded":"2024-09-22T03:30:23.245464+0000","last_fullsized":"2024-09-22T03:30:23.200129+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"232'565","reported_seq":787,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.146660+0000","last_change":"2024-09-22T03:30:23.272713+0000","last_active":"2024-09-22T03:30:26.146660+0000","last_peered":"2024-09-22T03:30:26.146660+0000","last_clean":"2024-09-22T03:29:50.378857+0000","last_became_active":"2024-09-22T03:30:23.272713+0000","last_became_peered":"2024-09-22T03:30:23.272713+0000","last_unstale":"2024-09-22T03:30:26.146660+0000","last_undegraded":"2024-09-22T03:30:23.217593+0000","last_fullsized":"2024-09-22T03:30:23.216950+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37093376,"num_objects":566,"num_object_clones":0,"num_object_copies":1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":566,"num_write_kb":36224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":5898240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":469,"reported_epoch":232,"state":"active+undersized","last_fresh":"2024-09-22T03:30:23.212025+0000","last_change":"2024-09-22T03:30:21.232982+0000","last_active":"2024-09-22T03:30:23.212025+0000","last_peered":"2024-09-22T03:30:23.212025+0000","last_clean":"2024-09-22T03:28:00.174365+0000","last_became_active":"2024-09-22T03:30:21.232982+0000","last_became_peered":"2024-09-22T03:30:21.232982+0000","last_unstale":"2024-09-22T03:30:23.212025+0000","last_undegraded":"2024-09-22T03:30:23.212025+0000","last_fullsized":"2024-09-22T03:30:21.202478+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"228'519","reported_seq":735,"reported_epoch":231,"state":"peering","last_fresh":"2024-09-22T03:30:21.199108+0000","last_change":"2024-09-22T03:30:20.176676+0000","last_active":"2024-09-22T03:30:20.176676+0000","last_peered":"2024-09-22T03:29:50.399409+0000","last_clean":"2024-09-22T03:29:50.399409+0000","last_became_active":"2024-09-22T03:29:37.372303+0000","last_became_peered":"2024-09-22T03:29:37.372303+0000","last_unstale":"2024-09-22T03:30:21.199108+0000","last_undegraded":"2024-09-22T03:30:21.199108+0000","last_fullsized":"2024-09-22T03:30:21.199108+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34013184,"num_objects":519,"num_object_clones":0,"num_object_copies":1557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":519,"num_write_kb":33216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":5963776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":594,"reported_epoch":231,"state":"peering","last_fresh":"2024-09-22T03:30:21.203192+0000","last_change":"2024-09-22T03:30:21.203192+0000","last_active":"2024-09-22T03:30:21.203192+0000","last_peered":"2024-09-22T03:30:20.182420+0000","last_clean":"2024-09-22T03:30:20.182420+0000","last_became_active":"2024-09-22T03:29:37.321358+0000","last_became_peered":"2024-09-22T03:29:37.321358+0000","last_unstale":"2024-09-22T03:30:21.203192+0000","last_undegraded":"2024-09-22T03:30:21.203192+0000","last_fullsized":"2024-09-22T03:30:21.203192+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"232'571","reported_seq":763,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-22T03:30:26.194094+0000","last_change":"2024-09-22T03:30:23.280082+0000","last_active":"2024-09-22T03:30:26.194094+0000","last_peered":"2024-09-22T03:30:26.194094+0000","last_clean":"2024-09-22T03:30:26.194094+0000","last_became_active":"2024-09-22T03:30:23.268764+0000","last_became_peered":"2024-09-22T03:30:23.268764+0000","last_unstale":"2024-09-22T03:30:26.194094+0000","last_undegraded":"2024-09-22T03:30:26.194094+0000","last_fullsized":"2024-09-22T03:30:26.194094+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37486592,"num_objects":572,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":572,"num_write_kb":36608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":4784128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":513,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:23.249272+0000","last_change":"2024-09-22T03:30:23.248893+0000","last_active":"2024-09-22T03:30:23.249272+0000","last_peered":"2024-09-22T03:30:23.249272+0000","last_clean":"2024-09-22T03:29:37.298796+0000","last_became_active":"2024-09-22T03:30:23.248893+0000","last_became_peered":"2024-09-22T03:30:23.248893+0000","last_unstale":"2024-09-22T03:30:23.249272+0000","last_undegraded":"2024-09-22T03:30:23.197485+0000","last_fullsized":"2024-09-22T03:30:23.196790+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":176,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:23.209832+0000","last_change":"2024-09-22T03:30:21.230836+0000","last_active":"2024-09-22T03:30:23.209832+0000","last_peered":"2024-09-22T03:30:23.209832+0000","last_clean":"2024-09-22T03:11:07.188537+0000","last_became_active":"2024-09-22T03:30:21.230836+0000","last_became_peered":"2024-09-22T03:30:21.230836+0000","last_unstale":"2024-09-22T03:30:23.209832+0000","last_undegraded":"2024-09-22T03:30:21.204361+0000","last_fullsized":"2024-09-22T03:30:21.203774+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"232'563","reported_seq":669,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.545032+0000","last_change":"2024-09-22T03:30:23.278710+0000","last_active":"2024-09-22T03:30:26.545032+0000","last_peered":"2024-09-22T03:30:26.545032+0000","last_clean":"2024-09-22T03:29:50.491443+0000","last_became_active":"2024-09-22T03:30:23.278710+0000","last_became_peered":"2024-09-22T03:30:23.278710+0000","last_unstale":"2024-09-22T03:30:26.545032+0000","last_undegraded":"2024-09-22T03:30:23.207055+0000","last_fullsized":"2024-09-22T03:30:23.206158+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36962304,"num_objects":564,"num_object_clones":0,"num_object_copies":1692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":564,"num_write_kb":36096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":564}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":365,"reported_epoch":232,"state":"active+undersized","last_fresh":"2024-09-22T03:30:23.206387+0000","last_change":"2024-09-22T03:30:21.234053+0000","last_active":"2024-09-22T03:30:23.206387+0000","last_peered":"2024-09-22T03:30:23.206387+0000","last_clean":"2024-09-22T03:29:37.202247+0000","last_became_active":"2024-09-22T03:30:21.234053+0000","last_became_peered":"2024-09-22T03:30:21.234053+0000","last_unstale":"2024-09-22T03:30:23.206387+0000","last_undegraded":"2024-09-22T03:30:23.206387+0000","last_fullsized":"2024-09-22T03:30:21.208404+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"232'577","reported_seq":702,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.563270+0000","last_change":"2024-09-22T03:30:23.238584+0000","last_active":"2024-09-22T03:30:26.563270+0000","last_peered":"2024-09-22T03:30:26.563270+0000","last_clean":"2024-09-22T03:29:50.363874+0000","last_became_active":"2024-09-22T03:30:23.238584+0000","last_became_peered":"2024-09-22T03:30:23.238584+0000","last_unstale":"2024-09-22T03:30:26.563270+0000","last_undegraded":"2024-09-22T03:30:23.198836+0000","last_fullsized":"2024-09-22T03:30:23.198324+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37879808,"num_objects":578,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":578,"num_write_kb":36992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":578}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":485,"reported_epoch":231,"state":"peering","last_fresh":"2024-09-22T03:30:21.187545+0000","last_change":"2024-09-22T03:30:20.171957+0000","last_active":"2024-09-22T03:30:20.171957+0000","last_peered":"2024-09-22T03:29:37.279264+0000","last_clean":"2024-09-22T03:29:37.279264+0000","last_became_active":"2024-09-22T03:29:37.276360+0000","last_became_peered":"2024-09-22T03:29:37.276360+0000","last_unstale":"2024-09-22T03:30:21.187545+0000","last_undegraded":"2024-09-22T03:30:21.187545+0000","last_fullsized":"2024-09-22T03:30:21.187545+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"228'486","reported_seq":699,"reported_epoch":231,"state":"peering","last_fresh":"2024-09-22T03:30:21.189609+0000","last_change":"2024-09-22T03:30:21.189609+0000","last_active":"2024-09-22T03:30:21.189609+0000","last_peered":"2024-09-22T03:30:20.172831+0000","last_clean":"2024-09-22T03:30:20.172831+0000","last_became_active":"2024-09-22T03:29:37.311970+0000","last_became_peered":"2024-09-22T03:29:37.311970+0000","last_unstale":"2024-09-22T03:30:21.189609+0000","last_undegraded":"2024-09-22T03:30:21.189609+0000","last_fullsized":"2024-09-22T03:30:21.189609+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":5701632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":443,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-22T03:30:23.273286+0000","last_change":"2024-09-22T03:30:23.273846+0000","last_active":"2024-09-22T03:30:23.273286+0000","last_peered":"2024-09-22T03:30:23.273286+0000","last_clean":"2024-09-22T03:30:20.198000+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:30:23.273286+0000","last_undegraded":"2024-09-22T03:30:23.273286+0000","last_fullsized":"2024-09-22T03:30:23.273286+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"232'548","reported_seq":761,"reported_epoch":232,"state":"active+undersized+degraded","last_fresh":"2024-09-22T03:30:26.212796+0000","last_change":"2024-09-22T03:30:23.258136+0000","last_active":"2024-09-22T03:30:26.212796+0000","last_peered":"2024-09-22T03:30:26.212796+0000","last_clean":"2024-09-22T03:29:50.422195+0000","last_became_active":"2024-09-22T03:30:23.258136+0000","last_became_peered":"2024-09-22T03:30:23.258136+0000","last_unstale":"2024-09-22T03:30:26.212796+0000","last_undegraded":"2024-09-22T03:30:23.205552+0000","last_fullsized":"2024-09-22T03:30:23.204835+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":549,"num_object_clones":0,"num_object_copies":1647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":549,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":549,"num_write_kb":35136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":5570560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":549}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":444,"reported_epoch":232,"state":"active+undersized","last_fresh":"2024-09-22T03:30:23.248976+0000","last_change":"2024-09-22T03:30:23.248582+0000","last_active":"2024-09-22T03:30:23.248976+0000","last_peered":"2024-09-22T03:30:23.248976+0000","last_clean":"2024-09-22T03:29:37.288137+0000","last_became_active":"2024-09-22T03:30:23.248582+0000","last_became_peered":"2024-09-22T03:30:23.248582+0000","last_unstale":"2024-09-22T03:30:23.248976+0000","last_undegraded":"2024-09-22T03:30:23.248976+0000","last_fullsized":"2024-09-22T03:30:23.199682+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":569704448,"num_objects":8693,"num_object_clones":0,"num_object_copies":26079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8693,"num_write_kb":556352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1260,"num_bytes_recovered":50659328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101580800,"data_stored":1625292800,"data_compressed":77174270,"data_compressed_allocated":101580800,"data_compressed_original":1625292800,"omap_allocated":0,"internal_metadata":0},"log_size":8682,"ondisk_log_size":8682,"up":37,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":18,"acting":18,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86576,"kb_used_data":35424,"kb_used_omap":0,"kb_used_meta":51136,"kb_avail":94285264,"statfs":{"total":96636764160,"available":96548110336,"internally_reserved":0,"allocated":36274176,"data_stored":550385188,"data_compressed":26096182,"data_compressed_allocated":34349056,"data_compressed_original":549584896,"omap_allocated":0,"internal_metadata":52363264},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79516,"kb_used_data":19532,"kb_used_omap":0,"kb_used_meta":59968,"kb_avail":94292324,"statfs":{"total":96636764160,"available":96555339776,"internally_reserved":0,"allocated":20000768,"data_stored":289745771,"data_compressed":13722166,"data_compressed_allocated":18063360,"data_compressed_original":288931840,"omap_allocated":0,"internal_metadata":61407232},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99532,"kb_used_data":31036,"kb_used_omap":0,"kb_used_meta":68480,"kb_avail":94272308,"statfs":{"total":96636764160,"available":96534843392,"internally_reserved":0,"allocated":31780864,"data_stored":478099524,"data_compressed":22665770,"data_compressed_allocated":29835264,"data_compressed_original":477282304,"omap_allocated":0,"internal_metadata":70123520},"hb_peers":[2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":204,"seq":876173328494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66080,"kb_used_data":19600,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":94305760,"statfs":{"total":96636764160,"available":96569098240,"internally_reserved":0,"allocated":20070400,"data_stored":291307569,"data_compressed":13796676,"data_compressed_allocated":18161664,"data_compressed_original":290504704,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18726912,"data_stored":299630592,"data_compressed":14227280,"data_compressed_allocated":18726912,"data_compressed_original":299630592,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29847552,"data_stored":477560832,"data_compressed":22676254,"data_compressed_allocated":29847552,"data_compressed_original":477560832,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18022400,"data_stored":288358400,"data_compressed":13692194,"data_compressed_allocated":18022400,"data_compressed_original":288358400,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34983936,"data_stored":559742976,"data_compressed":26578542,"data_compressed_allocated":34983936,"data_compressed_original":559742976,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:30:28.413 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 7.f 7.e 7.6 7.7 1.3 7.2 7.1 2024-09-22T03:30:28.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:28.540+0000 7f04a1c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:28.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:28.559+0000 7f04a1c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:28.775 INFO:teuthology.orchestra.run.smithi146.stderr:instructing pg(s) [7.1,7.7,7.e] on osd.1 to force-backfill; instructing pg(s) [1.3,7.2,7.6,7.f] on osd.3 to force-backfill; 2024-09-22T03:30:28.775 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:30:29.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 89 16 10092 10076 7.07502 22.5625 0.0279045 0.141303 2024-09-22T03:30:30.206 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:30.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout: 90 14 10466 10452 7.25749 23.5 0.0264388 0.137738 2024-09-22T03:30:30.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:30.333+0000 7f22822a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:30.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:30.338+0000 7f22822a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:30.343 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:30.386 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:30.386 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:31.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total time run: 90.0286 2024-09-22T03:30:31.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Total writes made: 10466 2024-09-22T03:30:31.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Write size: 65536 2024-09-22T03:30:31.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Object size: 65536 2024-09-22T03:30:31.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Bandwidth (MB/sec): 7.26575 2024-09-22T03:30:31.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Bandwidth: 10.3077 2024-09-22T03:30:31.306 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max bandwidth (MB/sec): 30 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min bandwidth (MB/sec): 0 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average IOPS: 116 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev IOPS: 164.929 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max IOPS: 480 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min IOPS: 0 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Average Latency(s): 0.137615 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Stddev Latency(s): 1.6327 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Max latency(s): 32.9313 2024-09-22T03:30:31.307 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Min latency(s): 0.00823066 2024-09-22T03:30:31.320 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:30:32.387 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:32.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:32.514+0000 7f75bd997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:32.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:32.520+0000 7f75bd997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:32.526 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:32.687 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:32.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:33.793 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 1, 3] 2024-09-22T03:30:33.794 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:30:33.794 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:33.794 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:33.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:33.940+0000 7f9f97a23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:33.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:33.950+0000 7f9f97a23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:33.955 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:33.958 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:33.959 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:33.959 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:34.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.073+0000 7f868e858640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.084+0000 7f868e858640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:30:34.107 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:30:34.111 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:34.111 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:34.111 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:34.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.228+0000 7f9c5c303640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.237+0000 7f9c5c303640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.253 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:30:34.254 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:30:34.257 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:34.258 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:34.258 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:34.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.369+0000 7fdac3f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.379+0000 7fdac3f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:config get 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:config help 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:config set ... 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:config show 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:deep_scrub 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_ops 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:dump_historic_slow_ops 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:dump_metrics [] 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:dump_ops_in_flight 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:dump_pgstate_history 2024-09-22T03:30:34.493 INFO:teuthology.orchestra.run.smithi146.stderr:Error EINVAL: invalid command 2024-09-22T03:30:34.496 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:34.496 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:34.689 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:34.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.824+0000 7f6272917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:34.829+0000 7f6272917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.836 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:35.069 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:35.069 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:37.070 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:37.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:37.207+0000 7f0a6e8ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:37.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:37.212+0000 7f0a6e8ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:37.219 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:37.411 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:37.412 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:39.412 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:39.497 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 1, 3] 2024-09-22T03:30:39.498 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:30:39.498 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-22T03:30:39.498 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:30:39.498 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:30:39.500 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:30:39.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:30:39.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:39.531+0000 7f3f3d197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:39.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:39.538+0000 7f3f3d197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:39.545 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:39.634 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:30:39.637 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:39.638 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:30:39.697 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:39.698 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:41.699 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:41.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:41.832+0000 7f98bdab2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:41.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:41.838+0000 7f98bdab2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:41.846 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:42.040 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:42.040 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:44.041 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:44.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:44.183+0000 7f7a1eed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:44.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:44.189+0000 7f7a1eed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:44.195 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:44.427 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:44.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:44.639 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:30:44.774 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:30:44.777 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:44.778 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:30:46.428 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:46.563+0000 7fe50ad97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:46.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:46.569+0000 7fe50ad97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:46.576 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:46.733 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:46.734 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:48.735 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:48.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:48.862+0000 7fa7c4d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:48.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:48.867+0000 7fa7c4d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:48.874 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:49.088 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:49.089 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:49.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:30:49.906 INFO:teuthology.orchestra.run.smithi146.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-22T03:30:49.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:49.909 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-22T03:30:51.091 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:51.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:51.226+0000 7fed33797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:51.231+0000 7fed33797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.237 INFO:teuthology.orchestra.run.smithi146.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-22T03:30:51.549 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-22T03:30:51.549 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-22T03:30:53.550 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:53.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:53.649+0000 7f5ef986c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:53.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:53.653+0000 7f5ef986c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:54.910 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: "ops_in_flight": [ 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 38, 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:30:55.013 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.290212+0000" 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.290227+0000" 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.290423+0000" 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.014 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.290582+0000" 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:30:55.015 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.313734+0000" 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.315739+0000" 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.016 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:54.195561+0000" 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:30:55.017 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.316467+0000" 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.018 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.316988+0000" 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.317716+0000" 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.021 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.022 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.023 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.024 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client_request", 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 40, 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "detail": { 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.025 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.292199+0000" 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.292206+0000" 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.026 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.028 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.028 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-22T03:30:55.029 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.292397+0000" 2024-09-22T03:30:55.029 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.029 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.029 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-22T03:30:55.030 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.314027+0000" 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.315981+0000" 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.031 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:54.195656+0000" 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-22T03:30:55.032 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.317560+0000" 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.033 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "2024-09-22T03:30:53.318510+0000" 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingLockOBC", 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.035 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::RecoverMissingSnaps", 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.036 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.037 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.038 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::WaitRepop", 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "blocking_event": { 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.039 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: "time_event": { 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: "initiated_at": "0.000000" 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-22T03:30:55.040 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:30:55.722 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:55.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:55.850+0000 7f4909597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:55.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:55.856+0000 7f4909597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:58.092 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:58.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:58.247+0000 7f6ac3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:58.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:30:58.254+0000 7f6ac3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:00.042 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 3, 0] 2024-09-22T03:31:00.042 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:31:00.042 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:00.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:00.195+0000 7f0e6fd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:00.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:00.208+0000 7f0e6fd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:00.493 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:00.494 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:00.511 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":827,"stamp":"2024-09-22T03:31:00.170115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97256011,"num_objects":1480,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":19597,"num_write_kb":671721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1399,"num_bytes_recovered":59768832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19543,"ondisk_log_size":19543,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":231280,"kb_used_data":44720,"kb_used_omap":0,"kb_used_meta":186496,"kb_avail":377256080,"statfs":{"total":386547056640,"available":386310225920,"internally_reserved":0,"allocated":45793280,"data_stored":609849765,"data_compressed":28809780,"data_compressed_allocated":37924864,"data_compressed_original":606552064,"omap_allocated":0,"internal_metadata":190971904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47382528,"num_objects":-723,"num_object_clones":0,"num_object_copies":-2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3851,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":9109504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.692413"},"pg_stats":[{"pgid":"7.f","version":"235'1206","reported_seq":1367,"reported_epoch":235,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.592986+0000","last_change":"2024-09-22T03:30:55.968687+0000","last_active":"2024-09-22T03:30:56.592986+0000","last_peered":"2024-09-22T03:30:56.592986+0000","last_clean":"2024-09-22T03:29:50.473527+0000","last_became_active":"2024-09-22T03:30:55.872819+0000","last_became_peered":"2024-09-22T03:30:55.872819+0000","last_unstale":"2024-09-22T03:30:56.592986+0000","last_undegraded":"2024-09-22T03:30:55.349606+0000","last_fullsized":"2024-09-22T03:30:55.243740+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1207,"num_write_kb":41024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":10}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"235'1194","reported_seq":1256,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.301303+0000","last_change":"2024-09-22T03:30:55.840942+0000","last_active":"2024-09-22T03:30:56.301303+0000","last_peered":"2024-09-22T03:30:56.301303+0000","last_clean":"2024-09-22T03:29:50.298165+0000","last_became_active":"2024-09-22T03:30:55.835682+0000","last_became_peered":"2024-09-22T03:30:55.835682+0000","last_unstale":"2024-09-22T03:30:56.301303+0000","last_undegraded":"2024-09-22T03:30:55.379184+0000","last_fullsized":"2024-09-22T03:30:55.342318+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1194,"log_dups_size":0,"ondisk_log_size":1194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":640,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1195,"num_write_kb":41472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":101}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"235'1228","reported_seq":1458,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:57.908291+0000","last_change":"2024-09-22T03:30:23.260902+0000","last_active":"2024-09-22T03:30:57.908291+0000","last_peered":"2024-09-22T03:30:57.908291+0000","last_clean":"2024-09-22T03:30:57.908291+0000","last_became_active":"2024-09-22T03:30:23.256730+0000","last_became_peered":"2024-09-22T03:30:23.256730+0000","last_unstale":"2024-09-22T03:30:57.908291+0000","last_undegraded":"2024-09-22T03:30:57.908291+0000","last_fullsized":"2024-09-22T03:30:57.908291+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1228,"log_dups_size":0,"ondisk_log_size":1228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636120,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1229,"num_write_kb":42049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":5636096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"235'1159","reported_seq":1362,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:57.751924+0000","last_change":"2024-09-22T03:30:23.282636+0000","last_active":"2024-09-22T03:30:57.751924+0000","last_peered":"2024-09-22T03:30:57.751924+0000","last_clean":"2024-09-22T03:30:57.751924+0000","last_became_active":"2024-09-22T03:30:23.274942+0000","last_became_peered":"2024-09-22T03:30:23.274942+0000","last_unstale":"2024-09-22T03:30:57.751924+0000","last_undegraded":"2024-09-22T03:30:57.751924+0000","last_fullsized":"2024-09-22T03:30:57.751924+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1159,"log_dups_size":0,"ondisk_log_size":1159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1160,"num_write_kb":39552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4849664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"235'1240","reported_seq":1397,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.574845+0000","last_change":"2024-09-22T03:30:55.842400+0000","last_active":"2024-09-22T03:30:56.574845+0000","last_peered":"2024-09-22T03:30:56.574845+0000","last_clean":"2024-09-22T03:29:50.461397+0000","last_became_active":"2024-09-22T03:30:55.836953+0000","last_became_peered":"2024-09-22T03:30:55.836953+0000","last_unstale":"2024-09-22T03:30:56.574845+0000","last_undegraded":"2024-09-22T03:30:55.241353+0000","last_fullsized":"2024-09-22T03:30:55.218236+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1240,"log_dups_size":0,"ondisk_log_size":1240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":656,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1241,"num_write_kb":43008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":103}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"235'1185","reported_seq":1322,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.501343+0000","last_change":"2024-09-22T03:30:55.876898+0000","last_active":"2024-09-22T03:30:56.501343+0000","last_peered":"2024-09-22T03:30:56.501343+0000","last_clean":"2024-09-22T03:29:50.455169+0000","last_became_active":"2024-09-22T03:30:55.864513+0000","last_became_peered":"2024-09-22T03:30:55.864513+0000","last_unstale":"2024-09-22T03:30:56.501343+0000","last_undegraded":"2024-09-22T03:30:55.397822+0000","last_fullsized":"2024-09-22T03:30:55.372901+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1185,"log_dups_size":0,"ondisk_log_size":1185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1186,"num_write_kb":41280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":104}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"235'1231","reported_seq":1465,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:57.931950+0000","last_change":"2024-09-22T03:30:23.264265+0000","last_active":"2024-09-22T03:30:57.931950+0000","last_peered":"2024-09-22T03:30:57.931950+0000","last_clean":"2024-09-22T03:30:57.931950+0000","last_became_active":"2024-09-22T03:30:23.260812+0000","last_became_peered":"2024-09-22T03:30:23.260812+0000","last_unstale":"2024-09-22T03:30:57.931950+0000","last_undegraded":"2024-09-22T03:30:57.931950+0000","last_fullsized":"2024-09-22T03:30:57.931950+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1232,"num_write_kb":41984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":5898240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"235'1207","reported_seq":1470,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.346828+0000","last_change":"2024-09-22T03:30:55.961204+0000","last_active":"2024-09-22T03:30:56.346828+0000","last_peered":"2024-09-22T03:30:56.346828+0000","last_clean":"2024-09-22T03:29:50.441556+0000","last_became_active":"2024-09-22T03:30:55.572703+0000","last_became_peered":"2024-09-22T03:30:55.572703+0000","last_unstale":"2024-09-22T03:30:56.346828+0000","last_undegraded":"2024-09-22T03:30:55.212500+0000","last_fullsized":"2024-09-22T03:30:55.188894+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":6750208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":86}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"235'1224","reported_seq":1404,"reported_epoch":235,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.607911+0000","last_change":"2024-09-22T03:30:55.914034+0000","last_active":"2024-09-22T03:30:56.607911+0000","last_peered":"2024-09-22T03:30:56.607911+0000","last_clean":"2024-09-22T03:29:50.442823+0000","last_became_active":"2024-09-22T03:30:55.854498+0000","last_became_peered":"2024-09-22T03:30:55.854498+0000","last_unstale":"2024-09-22T03:30:56.607911+0000","last_undegraded":"2024-09-22T03:30:55.351846+0000","last_fullsized":"2024-09-22T03:30:55.247763+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1224,"log_dups_size":0,"ondisk_log_size":1224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":81,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1224,"num_write_kb":41984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":7}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":420,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.246828+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:30:55.246828+0000","last_peered":"2024-09-22T03:30:55.246828+0000","last_clean":"2024-09-22T03:30:55.246828+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:30:55.246828+0000","last_undegraded":"2024-09-22T03:30:55.246828+0000","last_fullsized":"2024-09-22T03:30:55.246828+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"235'1208","reported_seq":1458,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.346102+0000","last_change":"2024-09-22T03:30:55.913220+0000","last_active":"2024-09-22T03:30:56.346102+0000","last_peered":"2024-09-22T03:30:56.346102+0000","last_clean":"2024-09-22T03:29:50.378857+0000","last_became_active":"2024-09-22T03:30:55.775473+0000","last_became_peered":"2024-09-22T03:30:55.775473+0000","last_unstale":"2024-09-22T03:30:56.346102+0000","last_undegraded":"2024-09-22T03:30:55.211807+0000","last_fullsized":"2024-09-22T03:30:55.189137+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1209,"num_write_kb":41664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":6291456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":93}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":480,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.189857+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:30:55.189857+0000","last_peered":"2024-09-22T03:30:55.189857+0000","last_clean":"2024-09-22T03:30:55.189857+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:30:55.189857+0000","last_undegraded":"2024-09-22T03:30:55.189857+0000","last_fullsized":"2024-09-22T03:30:55.189857+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"235'1298","reported_seq":1590,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:57.916635+0000","last_change":"2024-09-22T03:30:55.893472+0000","last_active":"2024-09-22T03:30:57.916635+0000","last_peered":"2024-09-22T03:30:57.916635+0000","last_clean":"2024-09-22T03:29:50.399409+0000","last_became_active":"2024-09-22T03:30:55.750416+0000","last_became_peered":"2024-09-22T03:30:55.750416+0000","last_unstale":"2024-09-22T03:30:57.916635+0000","last_undegraded":"2024-09-22T03:30:55.217977+0000","last_fullsized":"2024-09-22T03:30:55.191344+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1299,"num_write_kb":44352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":7798784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":87}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":612,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.220617+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:30:55.220617+0000","last_peered":"2024-09-22T03:30:55.220617+0000","last_clean":"2024-09-22T03:30:55.220617+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:30:55.220617+0000","last_undegraded":"2024-09-22T03:30:55.220617+0000","last_fullsized":"2024-09-22T03:30:55.220617+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"235'1227","reported_seq":1428,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:56.136475+0000","last_change":"2024-09-22T03:30:23.280082+0000","last_active":"2024-09-22T03:30:56.136475+0000","last_peered":"2024-09-22T03:30:56.136475+0000","last_clean":"2024-09-22T03:30:56.136475+0000","last_became_active":"2024-09-22T03:30:23.268764+0000","last_became_peered":"2024-09-22T03:30:23.268764+0000","last_unstale":"2024-09-22T03:30:56.136475+0000","last_undegraded":"2024-09-22T03:30:56.136475+0000","last_fullsized":"2024-09-22T03:30:56.136475+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1228,"num_write_kb":42496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":4784128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":526,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.216144+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:30:55.216144+0000","last_peered":"2024-09-22T03:30:55.216144+0000","last_clean":"2024-09-22T03:30:55.216144+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:30:55.216144+0000","last_undegraded":"2024-09-22T03:30:55.216144+0000","last_fullsized":"2024-09-22T03:30:55.216144+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":597,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.186291+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:30:55.186291+0000","last_peered":"2024-09-22T03:30:55.186291+0000","last_clean":"2024-09-22T03:30:55.186291+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:30:55.186291+0000","last_undegraded":"2024-09-22T03:30:55.186291+0000","last_fullsized":"2024-09-22T03:30:55.186291+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"235'1189","reported_seq":1323,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.382283+0000","last_change":"2024-09-22T03:30:55.868974+0000","last_active":"2024-09-22T03:30:56.382283+0000","last_peered":"2024-09-22T03:30:56.382283+0000","last_clean":"2024-09-22T03:29:50.491443+0000","last_became_active":"2024-09-22T03:30:55.859126+0000","last_became_peered":"2024-09-22T03:30:55.859126+0000","last_unstale":"2024-09-22T03:30:56.382283+0000","last_undegraded":"2024-09-22T03:30:55.370786+0000","last_fullsized":"2024-09-22T03:30:55.350508+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1189,"log_dups_size":0,"ondisk_log_size":1189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1190,"num_write_kb":40832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":86}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":378,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.341714+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:30:55.341714+0000","last_peered":"2024-09-22T03:30:55.341714+0000","last_clean":"2024-09-22T03:30:55.341714+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:30:55.341714+0000","last_undegraded":"2024-09-22T03:30:55.341714+0000","last_fullsized":"2024-09-22T03:30:55.341714+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"235'1240","reported_seq":1398,"reported_epoch":235,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.542102+0000","last_change":"2024-09-22T03:30:55.851643+0000","last_active":"2024-09-22T03:30:56.542102+0000","last_peered":"2024-09-22T03:30:56.542102+0000","last_clean":"2024-09-22T03:29:50.363874+0000","last_became_active":"2024-09-22T03:30:55.845441+0000","last_became_peered":"2024-09-22T03:30:55.845441+0000","last_unstale":"2024-09-22T03:30:56.542102+0000","last_undegraded":"2024-09-22T03:30:55.214055+0000","last_fullsized":"2024-09-22T03:30:55.187667+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1240,"log_dups_size":0,"ondisk_log_size":1240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7274496,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1241,"num_write_kb":43264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":111}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":504,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.189293+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:30:55.189293+0000","last_peered":"2024-09-22T03:30:55.189293+0000","last_clean":"2024-09-22T03:30:55.189293+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:30:55.189293+0000","last_undegraded":"2024-09-22T03:30:55.189293+0000","last_fullsized":"2024-09-22T03:30:55.189293+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"235'1220","reported_seq":1451,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:57.966761+0000","last_change":"2024-09-22T03:30:23.251701+0000","last_active":"2024-09-22T03:30:57.966761+0000","last_peered":"2024-09-22T03:30:57.966761+0000","last_clean":"2024-09-22T03:30:57.966761+0000","last_became_active":"2024-09-22T03:30:23.244706+0000","last_became_peered":"2024-09-22T03:30:23.244706+0000","last_unstale":"2024-09-22T03:30:57.966761+0000","last_undegraded":"2024-09-22T03:30:57.966761+0000","last_fullsized":"2024-09-22T03:30:57.966761+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1220,"log_dups_size":0,"ondisk_log_size":1220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1221,"num_write_kb":41728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":5701632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":453,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.341146+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:30:55.341146+0000","last_peered":"2024-09-22T03:30:55.341146+0000","last_clean":"2024-09-22T03:30:55.341146+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:30:55.341146+0000","last_undegraded":"2024-09-22T03:30:55.341146+0000","last_fullsized":"2024-09-22T03:30:55.341146+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"235'1187","reported_seq":1430,"reported_epoch":235,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-22T03:30:56.335614+0000","last_change":"2024-09-22T03:30:55.992418+0000","last_active":"2024-09-22T03:30:56.335614+0000","last_peered":"2024-09-22T03:30:56.335614+0000","last_clean":"2024-09-22T03:29:50.422195+0000","last_became_active":"2024-09-22T03:30:55.908668+0000","last_became_peered":"2024-09-22T03:30:55.908668+0000","last_unstale":"2024-09-22T03:30:56.335614+0000","last_undegraded":"2024-09-22T03:30:55.338424+0000","last_fullsized":"2024-09-22T03:30:55.188941+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1188,"num_write_kb":41728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6094848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":116}],"blocked_by":[],"up_primary":0,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":457,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-22T03:30:55.188513+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:30:55.188513+0000","last_peered":"2024-09-22T03:30:55.188513+0000","last_clean":"2024-09-22T03:30:55.188513+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:30:55.188513+0000","last_undegraded":"2024-09-22T03:30:55.188513+0000","last_fullsized":"2024-09-22T03:30:55.188513+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":96665624,"num_objects":1476,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19458,"num_write_kb":669825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1399,"num_bytes_recovered":59768832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33017856,"data_stored":528089160,"data_compressed":25075712,"data_compressed_allocated":33005568,"data_compressed_original":528089088,"omap_allocated":0,"internal_metadata":0},"log_size":19443,"ondisk_log_size":19443,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445410,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46752,"kb_used_data":8080,"kb_used_omap":0,"kb_used_meta":38656,"kb_avail":94325088,"statfs":{"total":96636764160,"available":96588890112,"internally_reserved":0,"allocated":8273920,"data_stored":101933788,"data_compressed":4801816,"data_compressed_allocated":6320128,"data_compressed_original":101122048,"omap_allocated":0,"internal_metadata":39583744},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568949995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56852,"kb_used_data":5188,"kb_used_omap":0,"kb_used_meta":51648,"kb_avail":94314988,"statfs":{"total":96636764160,"available":96578547712,"internally_reserved":0,"allocated":5312512,"data_stored":54158588,"data_compressed":2535132,"data_compressed_allocated":3338240,"data_compressed_original":53329920,"omap_allocated":0,"internal_metadata":52887552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648164998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71764,"kb_used_data":7428,"kb_used_omap":0,"kb_used_meta":64320,"kb_avail":94300076,"statfs":{"total":96636764160,"available":96563277824,"internally_reserved":0,"allocated":7606272,"data_stored":90858748,"data_compressed":4277852,"data_compressed_allocated":5632000,"data_compressed_original":90030080,"omap_allocated":0,"internal_metadata":65863680},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55912,"kb_used_data":24024,"kb_used_omap":0,"kb_used_meta":31872,"kb_avail":94315928,"statfs":{"total":96636764160,"available":96579510272,"internally_reserved":0,"allocated":24600576,"data_stored":362898641,"data_compressed":17194980,"data_compressed_allocated":22634496,"data_compressed_original":362070016,"omap_allocated":0,"internal_metadata":32636928},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18726912,"data_stored":299630592,"data_compressed":14227280,"data_compressed_allocated":18726912,"data_compressed_original":299630592,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5267456,"data_stored":84213784,"data_compressed":3998920,"data_compressed_allocated":5263360,"data_compressed_original":84213760,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3031040,"data_stored":48431128,"data_compressed":2299768,"data_compressed_allocated":3026944,"data_compressed_original":48431104,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5992448,"data_stored":95813656,"data_compressed":4549744,"data_compressed_allocated":5988352,"data_compressed_original":95813632,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:00.607 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:00.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:00.743+0000 7f34d8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:00.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:00.751+0000 7f34d8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:03.052 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:03.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:03.206+0000 7f8c0f658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:03.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:03.214+0000 7f8c0f658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:03.349 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Removed 10466 objects 2024-09-22T03:31:03.349 INFO:tasks.radosbench.radosbench.0.smithi146.stdout:Clean up completed and total clean up time :32.0283 2024-09-22T03:31:05.373 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:05.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:05.499+0000 7f80beb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:05.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:05.504+0000 7f80beb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:05.513 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 3, 0] 2024-09-22T03:31:05.514 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T03:31:05.514 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:05.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:05.646+0000 7f93ac957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:05.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:05.654+0000 7f93ac957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:06.011 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:06.011 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:06.029 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":830,"stamp":"2024-09-22T03:31:05.480463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19857995,"num_objects":299,"num_object_clones":0,"num_object_copies":897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":20778,"num_write_kb":671721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":115605504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20724,"ondisk_log_size":20724,"up":75,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":223356,"kb_used_data":32628,"kb_used_omap":0,"kb_used_meta":190656,"kb_avail":377264004,"statfs":{"total":386547056640,"available":386318340096,"internally_reserved":0,"allocated":33411072,"data_stored":411224485,"data_compressed":19377509,"data_compressed_allocated":25509888,"data_compressed_original":407912448,"omap_allocated":0,"internal_metadata":195231744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-124780544,"num_objects":-1904,"num_object_clones":0,"num_object_copies":-5712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1904,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1157,"num_bytes_recovered":64946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002257"},"pg_stats":[{"pgid":"7.f","version":"236'1264","reported_seq":1711,"reported_epoch":236,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:31:01.591242+0000","last_change":"2024-09-22T03:30:59.121274+0000","last_active":"2024-09-22T03:31:01.591242+0000","last_peered":"2024-09-22T03:31:01.591242+0000","last_clean":"2024-09-22T03:29:50.473527+0000","last_became_active":"2024-09-22T03:30:55.872819+0000","last_became_peered":"2024-09-22T03:30:55.872819+0000","last_unstale":"2024-09-22T03:31:01.591242+0000","last_undegraded":"2024-09-22T03:31:01.591242+0000","last_fullsized":"2024-09-22T03:30:55.243740+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1265,"num_write_kb":41024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":5505024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":17}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"236'1265","reported_seq":1471,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:01.312895+0000","last_change":"2024-09-22T03:30:55.840942+0000","last_active":"2024-09-22T03:31:01.312895+0000","last_peered":"2024-09-22T03:31:01.312895+0000","last_clean":"2024-09-22T03:29:50.298165+0000","last_became_active":"2024-09-22T03:30:55.835682+0000","last_became_peered":"2024-09-22T03:30:55.835682+0000","last_unstale":"2024-09-22T03:31:01.312895+0000","last_undegraded":"2024-09-22T03:30:55.379184+0000","last_fullsized":"2024-09-22T03:30:55.342318+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1266,"num_write_kb":41472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":5177344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"236'1310","reported_seq":1543,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:02.937560+0000","last_change":"2024-09-22T03:30:23.260902+0000","last_active":"2024-09-22T03:31:02.937560+0000","last_peered":"2024-09-22T03:31:02.937560+0000","last_clean":"2024-09-22T03:31:02.937560+0000","last_became_active":"2024-09-22T03:30:23.256730+0000","last_became_peered":"2024-09-22T03:30:23.256730+0000","last_unstale":"2024-09-22T03:31:02.937560+0000","last_undegraded":"2024-09-22T03:31:02.937560+0000","last_fullsized":"2024-09-22T03:31:02.937560+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1310,"log_dups_size":0,"ondisk_log_size":1310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262168,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1311,"num_write_kb":42049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":5636096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"236'1234","reported_seq":1440,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:02.840092+0000","last_change":"2024-09-22T03:30:23.282636+0000","last_active":"2024-09-22T03:31:02.840092+0000","last_peered":"2024-09-22T03:31:02.840092+0000","last_clean":"2024-09-22T03:31:02.840092+0000","last_became_active":"2024-09-22T03:30:23.274942+0000","last_became_peered":"2024-09-22T03:30:23.274942+0000","last_unstale":"2024-09-22T03:31:02.840092+0000","last_undegraded":"2024-09-22T03:31:02.840092+0000","last_fullsized":"2024-09-22T03:31:02.840092+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1234,"log_dups_size":0,"ondisk_log_size":1234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1235,"num_write_kb":39552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4849664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"236'1318","reported_seq":1633,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:01.555197+0000","last_change":"2024-09-22T03:30:55.842400+0000","last_active":"2024-09-22T03:31:01.555197+0000","last_peered":"2024-09-22T03:31:01.555197+0000","last_clean":"2024-09-22T03:29:50.461397+0000","last_became_active":"2024-09-22T03:30:55.836953+0000","last_became_peered":"2024-09-22T03:30:55.836953+0000","last_unstale":"2024-09-22T03:31:01.555197+0000","last_undegraded":"2024-09-22T03:30:55.241353+0000","last_fullsized":"2024-09-22T03:30:55.218236+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1319,"num_write_kb":43008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":6160384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":25}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"236'1263","reported_seq":1558,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:01.555219+0000","last_change":"2024-09-22T03:30:55.876898+0000","last_active":"2024-09-22T03:31:01.555219+0000","last_peered":"2024-09-22T03:31:01.555219+0000","last_clean":"2024-09-22T03:29:50.455169+0000","last_became_active":"2024-09-22T03:30:55.864513+0000","last_became_peered":"2024-09-22T03:30:55.864513+0000","last_unstale":"2024-09-22T03:31:01.555219+0000","last_undegraded":"2024-09-22T03:30:55.397822+0000","last_fullsized":"2024-09-22T03:30:55.372901+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1264,"num_write_kb":41280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":5636096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":26}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"236'1305","reported_seq":1542,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:02.625846+0000","last_change":"2024-09-22T03:30:23.264265+0000","last_active":"2024-09-22T03:31:02.625846+0000","last_peered":"2024-09-22T03:31:02.625846+0000","last_clean":"2024-09-22T03:31:02.625846+0000","last_became_active":"2024-09-22T03:30:23.260812+0000","last_became_peered":"2024-09-22T03:30:23.260812+0000","last_unstale":"2024-09-22T03:31:02.625846+0000","last_undegraded":"2024-09-22T03:31:02.625846+0000","last_fullsized":"2024-09-22T03:31:02.625846+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1305,"log_dups_size":0,"ondisk_log_size":1305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1306,"num_write_kb":41984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":5898240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"236'1269","reported_seq":1658,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:01.322885+0000","last_change":"2024-09-22T03:30:55.961204+0000","last_active":"2024-09-22T03:31:01.322885+0000","last_peered":"2024-09-22T03:31:01.322885+0000","last_clean":"2024-09-22T03:29:50.441556+0000","last_became_active":"2024-09-22T03:30:55.572703+0000","last_became_peered":"2024-09-22T03:30:55.572703+0000","last_unstale":"2024-09-22T03:31:01.322885+0000","last_undegraded":"2024-09-22T03:30:55.212500+0000","last_fullsized":"2024-09-22T03:30:55.188894+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1269,"log_dups_size":0,"ondisk_log_size":1269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1270,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":10813440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"236'1291","reported_seq":1747,"reported_epoch":236,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:31:01.591518+0000","last_change":"2024-09-22T03:30:59.481296+0000","last_active":"2024-09-22T03:31:01.591518+0000","last_peered":"2024-09-22T03:31:01.591518+0000","last_clean":"2024-09-22T03:29:50.442823+0000","last_became_active":"2024-09-22T03:30:55.854498+0000","last_became_peered":"2024-09-22T03:30:55.854498+0000","last_unstale":"2024-09-22T03:31:01.591518+0000","last_undegraded":"2024-09-22T03:31:01.591518+0000","last_fullsized":"2024-09-22T03:30:55.247763+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1291,"log_dups_size":0,"ondisk_log_size":1291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":41984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":6619136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":21}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":423,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.205922+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:31:00.205922+0000","last_peered":"2024-09-22T03:31:00.205922+0000","last_clean":"2024-09-22T03:31:00.205922+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:00.205922+0000","last_undegraded":"2024-09-22T03:31:00.205922+0000","last_fullsized":"2024-09-22T03:31:00.205922+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"236'1273","reported_seq":1655,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:01.296431+0000","last_change":"2024-09-22T03:30:55.913220+0000","last_active":"2024-09-22T03:31:01.296431+0000","last_peered":"2024-09-22T03:31:01.296431+0000","last_clean":"2024-09-22T03:29:50.378857+0000","last_became_active":"2024-09-22T03:30:55.775473+0000","last_became_peered":"2024-09-22T03:30:55.775473+0000","last_unstale":"2024-09-22T03:31:01.296431+0000","last_undegraded":"2024-09-22T03:30:55.211807+0000","last_fullsized":"2024-09-22T03:30:55.189137+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":579,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1274,"num_write_kb":41664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":10551296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":28}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":483,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.200405+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:31:00.200405+0000","last_peered":"2024-09-22T03:31:00.200405+0000","last_clean":"2024-09-22T03:31:00.200405+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:00.200405+0000","last_undegraded":"2024-09-22T03:31:00.200405+0000","last_fullsized":"2024-09-22T03:31:00.200405+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"236'1382","reported_seq":1844,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:02.979892+0000","last_change":"2024-09-22T03:30:55.893472+0000","last_active":"2024-09-22T03:31:02.979892+0000","last_peered":"2024-09-22T03:31:02.979892+0000","last_clean":"2024-09-22T03:29:50.399409+0000","last_became_active":"2024-09-22T03:30:55.750416+0000","last_became_peered":"2024-09-22T03:30:55.750416+0000","last_unstale":"2024-09-22T03:31:02.979892+0000","last_undegraded":"2024-09-22T03:30:55.217977+0000","last_fullsized":"2024-09-22T03:30:55.191344+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1382,"log_dups_size":0,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1383,"num_write_kb":44352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":13303808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":3}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":615,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.190690+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:31:00.190690+0000","last_peered":"2024-09-22T03:31:00.190690+0000","last_clean":"2024-09-22T03:31:00.190690+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:00.190690+0000","last_undegraded":"2024-09-22T03:31:00.190690+0000","last_fullsized":"2024-09-22T03:31:00.190690+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"236'1298","reported_seq":1502,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.222597+0000","last_change":"2024-09-22T03:30:23.280082+0000","last_active":"2024-09-22T03:31:01.222597+0000","last_peered":"2024-09-22T03:31:01.222597+0000","last_clean":"2024-09-22T03:31:01.222597+0000","last_became_active":"2024-09-22T03:30:23.268764+0000","last_became_peered":"2024-09-22T03:30:23.268764+0000","last_unstale":"2024-09-22T03:31:01.222597+0000","last_undegraded":"2024-09-22T03:31:01.222597+0000","last_fullsized":"2024-09-22T03:31:01.222597+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1299,"num_write_kb":42496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":4784128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":529,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.200315+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:31:00.200315+0000","last_peered":"2024-09-22T03:31:00.200315+0000","last_clean":"2024-09-22T03:31:00.200315+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:00.200315+0000","last_undegraded":"2024-09-22T03:31:00.200315+0000","last_fullsized":"2024-09-22T03:31:00.200315+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":600,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.199179+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:31:00.199179+0000","last_peered":"2024-09-22T03:31:00.199179+0000","last_clean":"2024-09-22T03:31:00.199179+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:00.199179+0000","last_undegraded":"2024-09-22T03:31:00.199179+0000","last_fullsized":"2024-09-22T03:31:00.199179+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2","version":"236'1253","reported_seq":1517,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:01.567569+0000","last_change":"2024-09-22T03:30:55.868974+0000","last_active":"2024-09-22T03:31:01.567569+0000","last_peered":"2024-09-22T03:31:01.567569+0000","last_clean":"2024-09-22T03:29:50.491443+0000","last_became_active":"2024-09-22T03:30:55.859126+0000","last_became_peered":"2024-09-22T03:30:55.859126+0000","last_unstale":"2024-09-22T03:31:01.567569+0000","last_undegraded":"2024-09-22T03:30:55.370786+0000","last_fullsized":"2024-09-22T03:30:55.350508+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1253,"log_dups_size":0,"ondisk_log_size":1253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1254,"num_write_kb":40832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":4784128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":381,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.209482+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:31:00.209482+0000","last_peered":"2024-09-22T03:31:00.209482+0000","last_clean":"2024-09-22T03:31:00.209482+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:00.209482+0000","last_undegraded":"2024-09-22T03:31:00.209482+0000","last_fullsized":"2024-09-22T03:31:00.209482+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3","version":"236'1328","reported_seq":1664,"reported_epoch":236,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-22T03:31:01.509901+0000","last_change":"2024-09-22T03:30:55.851643+0000","last_active":"2024-09-22T03:31:01.509901+0000","last_peered":"2024-09-22T03:31:01.509901+0000","last_clean":"2024-09-22T03:29:50.363874+0000","last_became_active":"2024-09-22T03:30:55.845441+0000","last_became_peered":"2024-09-22T03:30:55.845441+0000","last_unstale":"2024-09-22T03:31:01.509901+0000","last_undegraded":"2024-09-22T03:30:55.214055+0000","last_fullsized":"2024-09-22T03:30:55.187667+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1328,"log_dups_size":0,"ondisk_log_size":1328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1329,"num_write_kb":43264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":6488064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":23}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":507,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.186700+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:31:00.186700+0000","last_peered":"2024-09-22T03:31:00.186700+0000","last_clean":"2024-09-22T03:31:00.186700+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:00.186700+0000","last_undegraded":"2024-09-22T03:31:00.186700+0000","last_fullsized":"2024-09-22T03:31:00.186700+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"236'1299","reported_seq":1533,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:02.988350+0000","last_change":"2024-09-22T03:30:23.251701+0000","last_active":"2024-09-22T03:31:02.988350+0000","last_peered":"2024-09-22T03:31:02.988350+0000","last_clean":"2024-09-22T03:31:02.988350+0000","last_became_active":"2024-09-22T03:30:23.244706+0000","last_became_peered":"2024-09-22T03:30:23.244706+0000","last_unstale":"2024-09-22T03:31:02.988350+0000","last_undegraded":"2024-09-22T03:31:02.988350+0000","last_fullsized":"2024-09-22T03:31:02.988350+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1299,"log_dups_size":0,"ondisk_log_size":1299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1300,"num_write_kb":41728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":5701632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":456,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.211037+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:31:00.211037+0000","last_peered":"2024-09-22T03:31:00.211037+0000","last_clean":"2024-09-22T03:31:00.211037+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:00.211037+0000","last_undegraded":"2024-09-22T03:31:00.211037+0000","last_fullsized":"2024-09-22T03:31:00.211037+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"236'1272","reported_seq":1815,"reported_epoch":236,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-22T03:31:01.326894+0000","last_change":"2024-09-22T03:31:00.351059+0000","last_active":"2024-09-22T03:31:01.326894+0000","last_peered":"2024-09-22T03:31:01.326894+0000","last_clean":"2024-09-22T03:29:50.422195+0000","last_became_active":"2024-09-22T03:30:55.908668+0000","last_became_peered":"2024-09-22T03:30:55.908668+0000","last_unstale":"2024-09-22T03:31:01.326894+0000","last_undegraded":"2024-09-22T03:31:01.326894+0000","last_fullsized":"2024-09-22T03:30:55.188941+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:58.162328+0000","last_clean_scrub_stamp":"2024-09-22T03:28:58.162328+0000","objects_scrubbed":0,"log_size":1272,"log_dups_size":0,"ondisk_log_size":1272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1273,"num_write_kb":41728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":31}],"blocked_by":[],"up_primary":0,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":460,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.202481+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:31:00.202481+0000","last_peered":"2024-09-22T03:31:00.202481+0000","last_clean":"2024-09-22T03:31:00.202481+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:00.202481+0000","last_undegraded":"2024-09-22T03:31:00.202481+0000","last_fullsized":"2024-09-22T03:31:00.202481+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":19267608,"num_objects":295,"num_object_clones":0,"num_object_copies":885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20639,"num_write_kb":669825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":115605504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21811200,"data_stored":348782664,"data_compressed":16561280,"data_compressed_allocated":21798912,"data_compressed_original":348782592,"omap_allocated":0,"internal_metadata":0},"log_size":20624,"ondisk_log_size":20624,"up":48,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445415,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46544,"kb_used_data":3712,"kb_used_omap":0,"kb_used_meta":42816,"kb_avail":94325296,"statfs":{"total":96636764160,"available":96589103104,"internally_reserved":0,"allocated":3801088,"data_stored":30240988,"data_compressed":1397288,"data_compressed_allocated":1839104,"data_compressed_original":29425664,"omap_allocated":0,"internal_metadata":43843584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58248,"kb_used_data":2424,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313592,"statfs":{"total":96636764160,"available":96577118208,"internally_reserved":0,"allocated":2482176,"data_stored":8745724,"data_compressed":378516,"data_compressed_allocated":499712,"data_compressed_original":7913472,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55988,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":51840,"kb_avail":94315852,"statfs":{"total":96636764160,"available":96579432448,"internally_reserved":0,"allocated":4231168,"data_stored":36729596,"data_compressed":1707340,"data_compressed_allocated":2248704,"data_compressed_original":35897344,"omap_allocated":0,"internal_metadata":53084160},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62576,"kb_used_data":22360,"kb_used_omap":0,"kb_used_meta":40192,"kb_avail":94309264,"statfs":{"total":96636764160,"available":96572686336,"internally_reserved":0,"allocated":22896640,"data_stored":335508177,"data_compressed":15894365,"data_compressed_allocated":20922368,"data_compressed_original":334675968,"omap_allocated":0,"internal_metadata":41156608},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18726912,"data_stored":299630592,"data_compressed":14227280,"data_compressed_allocated":18726912,"data_compressed_original":299630592,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1462272,"data_stored":23330840,"data_compressed":1107872,"data_compressed_allocated":1458176,"data_compressed_original":23330816,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":2293784,"data_compressed":108920,"data_compressed_allocated":143360,"data_compressed_original":2293760,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1474560,"data_stored":23527448,"data_compressed":1117208,"data_compressed_allocated":1470464,"data_compressed_original":23527424,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:06.030 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 7.e 7.b 7.a 7.2 7.3 2024-09-22T03:31:06.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:06.206+0000 7f0a436d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:06.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:06.223+0000 7f0a436d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:06.527 INFO:teuthology.orchestra.run.smithi146.stderr:instructing pg(s) [7.e] on osd.1 to force-backfill; instructing pg(s) [7.2,7.3,7.a,7.b] on osd.3 to force-backfill; 2024-09-22T03:31:06.527 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:31:07.629 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:07.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:07.742+0000 7ffb94757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:07.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:07.747+0000 7ffb94757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:08.369 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-22T03:31:08.369 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-22T03:31:08.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:08.482+0000 7f77c5957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:08.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:08.495+0000 7f77c5957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'unique_pool_4' removed 2024-09-22T03:31:09.242 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-22T03:31:09.250 INFO:tasks.thrashosds:joining thrashosds 2024-09-22T03:31:11.543 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-22T03:31:11.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:11.652+0000 7faa02d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:11.658 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:11.656+0000 7faa02d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:11.671 INFO:teuthology.orchestra.run.smithi146.stdout:{} 2024-09-22T03:31:11.671 INFO:teuthology.orchestra.run.smithi146.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T03:31:11.682 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:31:11.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:11.780+0000 7f4d004c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:11.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:11.789+0000 7f4d004c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:12.235 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:31:12.255 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-22T03:31:12.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:12.354+0000 7f2ee4797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:12.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:12.363+0000 7f2ee4797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.236 INFO:teuthology.orchestra.run.smithi146.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-22T03:31:13.255 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:31:13.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:13.357+0000 7feec402c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:13.365+0000 7feec402c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:14.241 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:31:14.260 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-22T03:31:14.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:14.359+0000 7fe96a70e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:14.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:14.368+0000 7fe96a70e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:15.173 INFO:teuthology.orchestra.run.smithi146.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-22T03:31:15.191 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:31:15.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:15.290+0000 7f3af8b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:15.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:15.298+0000 7f3af8b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:16.246 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:31:16.265 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-22T03:31:16.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:16.366+0000 7f4607997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:16.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:16.375+0000 7f4607997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:17.250 INFO:teuthology.orchestra.run.smithi146.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-22T03:31:17.269 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:31:17.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:17.371+0000 7fbdb506c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:17.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:17.379+0000 7fbdb506c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:18.254 INFO:teuthology.orchestra.run.smithi146.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:31:18.274 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-22T03:31:18.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:18.376+0000 7fe8edd0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:18.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:18.385+0000 7fe8edd0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:19.260 INFO:teuthology.orchestra.run.smithi146.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-22T03:31:19.277 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-22T03:31:19.277 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T03:31:19.278 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:31:19.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.381+0000 7f8ef6d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:19.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.388+0000 7f8ef6d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:19.568 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:19.568 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":248,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:31:19.258843+0000","last_up_change":"2024-09-22T03:30:53.170699+0000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-22T03:11:05.202857+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-22T03:11:05.252894+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":204,"last_clean_end":228,"up_from":233,"up_thru":237,"down_at":229,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3847282613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3847282613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3847282613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":3847282613}]},"public_addr":"172.21.15.146:6804/3847282613","cluster_addr":"172.21.15.146:6805/3847282613","heartbeat_back_addr":"172.21.15.146:6812/3847282613","heartbeat_front_addr":"172.21.15.146:6811/3847282613","state":["exists","up"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":159,"last_clean_end":206,"up_from":209,"up_thru":236,"down_at":207,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":2584126373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2584126373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2584126373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":2584126373}]},"public_addr":"172.21.15.146:6801/2584126373","cluster_addr":"172.21.15.146:6800/2584126373","heartbeat_back_addr":"172.21.15.146:6802/2584126373","heartbeat_front_addr":"172.21.15.146:6803/2584126373","state":["exists","up"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":178,"last_clean_end":180,"up_from":185,"up_thru":234,"down_at":183,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2090648228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6805","nonce":2090648228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":2090648228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2090648228}]},"public_addr":"172.21.15.148:6806/2090648228","cluster_addr":"172.21.15.148:6805/2090648228","heartbeat_back_addr":"172.21.15.148:6809/2090648228","heartbeat_front_addr":"172.21.15.148:6804/2090648228","state":["exists","up"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":227,"last_clean_end":228,"up_from":231,"up_thru":238,"down_at":230,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6811","nonce":1614916954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1614916954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6807","nonce":1614916954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1614916954}]},"public_addr":"172.21.15.148:6811/1614916954","cluster_addr":"172.21.15.148:6810/1614916954","heartbeat_back_addr":"172.21.15.148:6807/1614916954","heartbeat_front_addr":"172.21.15.148:6808/1614916954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-22T03:30:20.161661+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"2024-09-22T03:28:32.148741+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"2024-09-22T03:26:48.178527+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"2024-09-22T03:30:21.173762+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:31:19.581 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T03:31:19.582 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:31:19.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.682+0000 7f0594f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:19.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.688+0000 7f0594f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:19.869 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:19.869 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":248,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:31:19.258843+0000","last_up_change":"2024-09-22T03:30:53.170699+0000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-22T03:11:05.202857+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-22T03:11:05.252894+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":204,"last_clean_end":228,"up_from":233,"up_thru":237,"down_at":229,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3847282613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3847282613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3847282613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":3847282613}]},"public_addr":"172.21.15.146:6804/3847282613","cluster_addr":"172.21.15.146:6805/3847282613","heartbeat_back_addr":"172.21.15.146:6812/3847282613","heartbeat_front_addr":"172.21.15.146:6811/3847282613","state":["exists","up"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":159,"last_clean_end":206,"up_from":209,"up_thru":236,"down_at":207,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":2584126373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2584126373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2584126373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":2584126373}]},"public_addr":"172.21.15.146:6801/2584126373","cluster_addr":"172.21.15.146:6800/2584126373","heartbeat_back_addr":"172.21.15.146:6802/2584126373","heartbeat_front_addr":"172.21.15.146:6803/2584126373","state":["exists","up"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":178,"last_clean_end":180,"up_from":185,"up_thru":234,"down_at":183,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2090648228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6805","nonce":2090648228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":2090648228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2090648228}]},"public_addr":"172.21.15.148:6806/2090648228","cluster_addr":"172.21.15.148:6805/2090648228","heartbeat_back_addr":"172.21.15.148:6809/2090648228","heartbeat_front_addr":"172.21.15.148:6804/2090648228","state":["exists","up"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":227,"last_clean_end":228,"up_from":231,"up_thru":238,"down_at":230,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6811","nonce":1614916954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1614916954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6807","nonce":1614916954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1614916954}]},"public_addr":"172.21.15.148:6811/1614916954","cluster_addr":"172.21.15.148:6810/1614916954","heartbeat_back_addr":"172.21.15.148:6807/1614916954","heartbeat_front_addr":"172.21.15.148:6808/1614916954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-22T03:30:20.161661+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"2024-09-22T03:28:32.148741+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"2024-09-22T03:26:48.178527+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"2024-09-22T03:30:21.173762+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:31:19.883 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T03:31:19.883 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T03:31:19.884 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T03:31:19.884 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T03:31:20.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.995+0000 7fc701b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.995+0000 7f191ff0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.996+0000 7f5135197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:19.996+0000 7f43c9f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.003+0000 7fc701b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.004+0000 7f191ff0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.004+0000 7f43c9f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.004+0000 7f5135197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.038 INFO:teuthology.orchestra.run.smithi146.stdout:1000727379994 2024-09-22T03:31:20.039 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:31:20.039 INFO:teuthology.orchestra.run.smithi146.stdout:992137445433 2024-09-22T03:31:20.040 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:31:20.043 INFO:teuthology.orchestra.run.smithi146.stdout:897648165022 2024-09-22T03:31:20.043 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:31:20.044 INFO:teuthology.orchestra.run.smithi146.stdout:794568950017 2024-09-22T03:31:20.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:31:20.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.146+0000 7f2ba9197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.158+0000 7f2ba9197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.193+0000 7f7b8f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.193+0000 7f50b9957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.195+0000 7f994f397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.201+0000 7f50b9957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.202+0000 7f7b8f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:20.204+0000 7f994f397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.361 INFO:teuthology.orchestra.run.smithi146.stdout:1000727379993 2024-09-22T03:31:20.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 1000727379994 got 1000727379993 for osd.0 2024-09-22T03:31:20.411 INFO:teuthology.orchestra.run.smithi146.stdout:897648165018 2024-09-22T03:31:20.419 INFO:teuthology.orchestra.run.smithi146.stdout:794568950015 2024-09-22T03:31:20.420 INFO:teuthology.orchestra.run.smithi146.stdout:992137445430 2024-09-22T03:31:20.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 897648165022 got 897648165018 for osd.1 2024-09-22T03:31:20.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 794568950017 got 794568950015 for osd.2 2024-09-22T03:31:20.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 992137445433 got 992137445430 for osd.3 2024-09-22T03:31:21.376 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:31:21.426 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:31:21.436 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:31:21.436 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:31:21.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.495+0000 7f9f98f0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.505+0000 7f9f98f0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.538+0000 7f55bf6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.550+0000 7f55bf6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.579+0000 7fea70757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.579+0000 7fb8b7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.587+0000 7fb8b7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:21.588+0000 7fea70757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.712 INFO:teuthology.orchestra.run.smithi146.stdout:1000727379993 2024-09-22T03:31:21.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 1000727379994 got 1000727379993 for osd.0 2024-09-22T03:31:21.746 INFO:teuthology.orchestra.run.smithi146.stdout:897648165018 2024-09-22T03:31:21.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 897648165022 got 897648165018 for osd.1 2024-09-22T03:31:21.787 INFO:teuthology.orchestra.run.smithi146.stdout:992137445430 2024-09-22T03:31:21.789 INFO:teuthology.orchestra.run.smithi146.stdout:794568950015 2024-09-22T03:31:21.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 992137445433 got 992137445430 for osd.3 2024-09-22T03:31:21.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 794568950017 got 794568950015 for osd.2 2024-09-22T03:31:22.732 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:31:22.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:31:22.803 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:31:22.803 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:31:22.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.851+0000 7fd0aa597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.862+0000 7fd0aa597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.889+0000 7f275af97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.901+0000 7f275af97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.930+0000 7f04caa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.932+0000 7f83a4f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.938+0000 7f04caa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:22.941+0000 7f83a4f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:23.071 INFO:teuthology.orchestra.run.smithi146.stdout:1000727379994 2024-09-22T03:31:23.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 1000727379994 got 1000727379994 for osd.0 2024-09-22T03:31:23.086 DEBUG:teuthology.parallel:result is None 2024-09-22T03:31:23.095 INFO:teuthology.orchestra.run.smithi146.stdout:897648165023 2024-09-22T03:31:23.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 897648165022 got 897648165023 for osd.1 2024-09-22T03:31:23.108 DEBUG:teuthology.parallel:result is None 2024-09-22T03:31:23.132 INFO:teuthology.orchestra.run.smithi146.stdout:794568950017 2024-09-22T03:31:23.142 INFO:teuthology.orchestra.run.smithi146.stdout:992137445433 2024-09-22T03:31:23.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 794568950017 got 794568950017 for osd.2 2024-09-22T03:31:23.146 DEBUG:teuthology.parallel:result is None 2024-09-22T03:31:23.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 992137445433 got 992137445433 for osd.3 2024-09-22T03:31:23.156 DEBUG:teuthology.parallel:result is None 2024-09-22T03:31:23.156 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:31:23.156 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:23.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:23.298+0000 7f9f9d757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:23.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:23.304+0000 7f9f9d757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:23.481 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:23.482 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:23.494 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":850,"stamp":"2024-09-22T03:31:21.483649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":216976,"kb_used_data":8208,"kb_used_omap":0,"kb_used_meta":208704,"kb_avail":377270384,"statfs":{"total":386547056640,"available":386324873216,"internally_reserved":0,"allocated":8404992,"data_stored":5173761,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213712896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.237260"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":459,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268139+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:31:19.268139+0000","last_peered":"2024-09-22T03:31:19.268139+0000","last_clean":"2024-09-22T03:31:19.268139+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:19.268139+0000","last_undegraded":"2024-09-22T03:31:19.268139+0000","last_fullsized":"2024-09-22T03:31:19.268139+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":519,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.278193+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:31:19.278193+0000","last_peered":"2024-09-22T03:31:19.278193+0000","last_clean":"2024-09-22T03:31:19.278193+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:19.278193+0000","last_undegraded":"2024-09-22T03:31:19.278193+0000","last_fullsized":"2024-09-22T03:31:19.278193+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":651,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268560+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:31:19.268560+0000","last_peered":"2024-09-22T03:31:19.268560+0000","last_clean":"2024-09-22T03:31:19.268560+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:19.268560+0000","last_undegraded":"2024-09-22T03:31:19.268560+0000","last_fullsized":"2024-09-22T03:31:19.268560+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":565,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266102+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:31:19.266102+0000","last_peered":"2024-09-22T03:31:19.266102+0000","last_clean":"2024-09-22T03:31:19.266102+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:19.266102+0000","last_undegraded":"2024-09-22T03:31:19.266102+0000","last_fullsized":"2024-09-22T03:31:19.266102+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":636,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.277105+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:31:19.277105+0000","last_peered":"2024-09-22T03:31:19.277105+0000","last_clean":"2024-09-22T03:31:19.277105+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:19.277105+0000","last_undegraded":"2024-09-22T03:31:19.277105+0000","last_fullsized":"2024-09-22T03:31:19.277105+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":417,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.297740+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:31:19.297740+0000","last_peered":"2024-09-22T03:31:19.297740+0000","last_clean":"2024-09-22T03:31:19.297740+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:19.297740+0000","last_undegraded":"2024-09-22T03:31:19.297740+0000","last_fullsized":"2024-09-22T03:31:19.297740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":543,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266372+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:31:19.266372+0000","last_peered":"2024-09-22T03:31:19.266372+0000","last_clean":"2024-09-22T03:31:19.266372+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:19.266372+0000","last_undegraded":"2024-09-22T03:31:19.266372+0000","last_fullsized":"2024-09-22T03:31:19.266372+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":492,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.298195+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:31:19.298195+0000","last_peered":"2024-09-22T03:31:19.298195+0000","last_clean":"2024-09-22T03:31:19.298195+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:19.298195+0000","last_undegraded":"2024-09-22T03:31:19.298195+0000","last_fullsized":"2024-09-22T03:31:19.298195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":496,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.296313+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:31:19.296313+0000","last_peered":"2024-09-22T03:31:19.296313+0000","last_clean":"2024-09-22T03:31:19.296313+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:19.296313+0000","last_undegraded":"2024-09-22T03:31:19.296313+0000","last_fullsized":"2024-09-22T03:31:19.296313+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62884,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94308956,"statfs":{"total":96636764160,"available":96572370944,"internally_reserved":0,"allocated":2052096,"data_stored":849207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57896,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313944,"statfs":{"total":96636764160,"available":96577478656,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58088,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":56000,"kb_avail":94313752,"statfs":{"total":96636764160,"available":96577282048,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57344000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38108,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94333732,"statfs":{"total":96636764160,"available":96597741568,"internally_reserved":0,"allocated":2109440,"data_stored":1439556,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:23.495 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:23.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:23.595+0000 7fd874197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:23.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:23.601+0000 7fd874197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:23.778 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:23.858 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:23.858 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":851,"stamp":"2024-09-22T03:31:23.484005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":216992,"kb_used_data":8224,"kb_used_omap":0,"kb_used_meta":208704,"kb_avail":377270368,"statfs":{"total":386547056640,"available":386324856832,"internally_reserved":0,"allocated":8421376,"data_stored":5179609,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213712896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.233831"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":459,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268139+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:31:19.268139+0000","last_peered":"2024-09-22T03:31:19.268139+0000","last_clean":"2024-09-22T03:31:19.268139+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:19.268139+0000","last_undegraded":"2024-09-22T03:31:19.268139+0000","last_fullsized":"2024-09-22T03:31:19.268139+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":519,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.278193+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:31:19.278193+0000","last_peered":"2024-09-22T03:31:19.278193+0000","last_clean":"2024-09-22T03:31:19.278193+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:19.278193+0000","last_undegraded":"2024-09-22T03:31:19.278193+0000","last_fullsized":"2024-09-22T03:31:19.278193+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":651,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268560+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:31:19.268560+0000","last_peered":"2024-09-22T03:31:19.268560+0000","last_clean":"2024-09-22T03:31:19.268560+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:19.268560+0000","last_undegraded":"2024-09-22T03:31:19.268560+0000","last_fullsized":"2024-09-22T03:31:19.268560+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":565,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266102+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:31:19.266102+0000","last_peered":"2024-09-22T03:31:19.266102+0000","last_clean":"2024-09-22T03:31:19.266102+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:19.266102+0000","last_undegraded":"2024-09-22T03:31:19.266102+0000","last_fullsized":"2024-09-22T03:31:19.266102+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":636,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.277105+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:31:19.277105+0000","last_peered":"2024-09-22T03:31:19.277105+0000","last_clean":"2024-09-22T03:31:19.277105+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:19.277105+0000","last_undegraded":"2024-09-22T03:31:19.277105+0000","last_fullsized":"2024-09-22T03:31:19.277105+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":417,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.297740+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:31:19.297740+0000","last_peered":"2024-09-22T03:31:19.297740+0000","last_clean":"2024-09-22T03:31:19.297740+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:19.297740+0000","last_undegraded":"2024-09-22T03:31:19.297740+0000","last_fullsized":"2024-09-22T03:31:19.297740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":543,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266372+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:31:19.266372+0000","last_peered":"2024-09-22T03:31:19.266372+0000","last_clean":"2024-09-22T03:31:19.266372+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:19.266372+0000","last_undegraded":"2024-09-22T03:31:19.266372+0000","last_fullsized":"2024-09-22T03:31:19.266372+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":492,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.298195+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:31:19.298195+0000","last_peered":"2024-09-22T03:31:19.298195+0000","last_clean":"2024-09-22T03:31:19.298195+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:19.298195+0000","last_undegraded":"2024-09-22T03:31:19.298195+0000","last_fullsized":"2024-09-22T03:31:19.298195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":496,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.296313+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:31:19.296313+0000","last_peered":"2024-09-22T03:31:19.296313+0000","last_clean":"2024-09-22T03:31:19.296313+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:19.296313+0000","last_undegraded":"2024-09-22T03:31:19.296313+0000","last_fullsized":"2024-09-22T03:31:19.296313+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445435,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62892,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94308948,"statfs":{"total":96636764160,"available":96572362752,"internally_reserved":0,"allocated":2060288,"data_stored":852131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57896,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313944,"statfs":{"total":96636764160,"available":96577478656,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58088,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":56000,"kb_avail":94313752,"statfs":{"total":96636764160,"available":96577282048,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57344000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38116,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2117632,"data_stored":1442480,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:23.859 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:31:23.859 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-22T03:31:23.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:23.966+0000 7f4a50197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:23.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:23.975+0000 7f4a50197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.170 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-22T03:31:24.224 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T03:31:24.224 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:24.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:24.326+0000 7fdfc2957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:24.333+0000 7fdfc2957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.510 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:24.511 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:24.524 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":851,"stamp":"2024-09-22T03:31:23.484005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":216992,"kb_used_data":8224,"kb_used_omap":0,"kb_used_meta":208704,"kb_avail":377270368,"statfs":{"total":386547056640,"available":386324856832,"internally_reserved":0,"allocated":8421376,"data_stored":5179609,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213712896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.233831"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":459,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268139+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:31:19.268139+0000","last_peered":"2024-09-22T03:31:19.268139+0000","last_clean":"2024-09-22T03:31:19.268139+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:19.268139+0000","last_undegraded":"2024-09-22T03:31:19.268139+0000","last_fullsized":"2024-09-22T03:31:19.268139+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":519,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.278193+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:31:19.278193+0000","last_peered":"2024-09-22T03:31:19.278193+0000","last_clean":"2024-09-22T03:31:19.278193+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:19.278193+0000","last_undegraded":"2024-09-22T03:31:19.278193+0000","last_fullsized":"2024-09-22T03:31:19.278193+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":651,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268560+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:31:19.268560+0000","last_peered":"2024-09-22T03:31:19.268560+0000","last_clean":"2024-09-22T03:31:19.268560+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:19.268560+0000","last_undegraded":"2024-09-22T03:31:19.268560+0000","last_fullsized":"2024-09-22T03:31:19.268560+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":565,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266102+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:31:19.266102+0000","last_peered":"2024-09-22T03:31:19.266102+0000","last_clean":"2024-09-22T03:31:19.266102+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:19.266102+0000","last_undegraded":"2024-09-22T03:31:19.266102+0000","last_fullsized":"2024-09-22T03:31:19.266102+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":636,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.277105+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:31:19.277105+0000","last_peered":"2024-09-22T03:31:19.277105+0000","last_clean":"2024-09-22T03:31:19.277105+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:19.277105+0000","last_undegraded":"2024-09-22T03:31:19.277105+0000","last_fullsized":"2024-09-22T03:31:19.277105+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":417,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.297740+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:31:19.297740+0000","last_peered":"2024-09-22T03:31:19.297740+0000","last_clean":"2024-09-22T03:31:19.297740+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:19.297740+0000","last_undegraded":"2024-09-22T03:31:19.297740+0000","last_fullsized":"2024-09-22T03:31:19.297740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":543,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266372+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:31:19.266372+0000","last_peered":"2024-09-22T03:31:19.266372+0000","last_clean":"2024-09-22T03:31:19.266372+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:19.266372+0000","last_undegraded":"2024-09-22T03:31:19.266372+0000","last_fullsized":"2024-09-22T03:31:19.266372+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":492,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.298195+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:31:19.298195+0000","last_peered":"2024-09-22T03:31:19.298195+0000","last_clean":"2024-09-22T03:31:19.298195+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:19.298195+0000","last_undegraded":"2024-09-22T03:31:19.298195+0000","last_fullsized":"2024-09-22T03:31:19.298195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":496,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.296313+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:31:19.296313+0000","last_peered":"2024-09-22T03:31:19.296313+0000","last_clean":"2024-09-22T03:31:19.296313+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:19.296313+0000","last_undegraded":"2024-09-22T03:31:19.296313+0000","last_fullsized":"2024-09-22T03:31:19.296313+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445435,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62892,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94308948,"statfs":{"total":96636764160,"available":96572362752,"internally_reserved":0,"allocated":2060288,"data_stored":852131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57896,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313944,"statfs":{"total":96636764160,"available":96577478656,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58088,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":56000,"kb_avail":94313752,"statfs":{"total":96636764160,"available":96577282048,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57344000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38116,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2117632,"data_stored":1442480,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:24.525 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:24.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:24.625+0000 7f469ced7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:24.632+0000 7f469ced7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.807 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:24.808 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:24.820 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":851,"stamp":"2024-09-22T03:31:23.484005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":216992,"kb_used_data":8224,"kb_used_omap":0,"kb_used_meta":208704,"kb_avail":377270368,"statfs":{"total":386547056640,"available":386324856832,"internally_reserved":0,"allocated":8421376,"data_stored":5179609,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213712896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.233831"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":459,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268139+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:31:19.268139+0000","last_peered":"2024-09-22T03:31:19.268139+0000","last_clean":"2024-09-22T03:31:19.268139+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:19.268139+0000","last_undegraded":"2024-09-22T03:31:19.268139+0000","last_fullsized":"2024-09-22T03:31:19.268139+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":519,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.278193+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:31:19.278193+0000","last_peered":"2024-09-22T03:31:19.278193+0000","last_clean":"2024-09-22T03:31:19.278193+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:19.278193+0000","last_undegraded":"2024-09-22T03:31:19.278193+0000","last_fullsized":"2024-09-22T03:31:19.278193+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":651,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268560+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:31:19.268560+0000","last_peered":"2024-09-22T03:31:19.268560+0000","last_clean":"2024-09-22T03:31:19.268560+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:19.268560+0000","last_undegraded":"2024-09-22T03:31:19.268560+0000","last_fullsized":"2024-09-22T03:31:19.268560+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":565,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266102+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:31:19.266102+0000","last_peered":"2024-09-22T03:31:19.266102+0000","last_clean":"2024-09-22T03:31:19.266102+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:19.266102+0000","last_undegraded":"2024-09-22T03:31:19.266102+0000","last_fullsized":"2024-09-22T03:31:19.266102+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":636,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.277105+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:31:19.277105+0000","last_peered":"2024-09-22T03:31:19.277105+0000","last_clean":"2024-09-22T03:31:19.277105+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:19.277105+0000","last_undegraded":"2024-09-22T03:31:19.277105+0000","last_fullsized":"2024-09-22T03:31:19.277105+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":417,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.297740+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:31:19.297740+0000","last_peered":"2024-09-22T03:31:19.297740+0000","last_clean":"2024-09-22T03:31:19.297740+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:19.297740+0000","last_undegraded":"2024-09-22T03:31:19.297740+0000","last_fullsized":"2024-09-22T03:31:19.297740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":543,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266372+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:31:19.266372+0000","last_peered":"2024-09-22T03:31:19.266372+0000","last_clean":"2024-09-22T03:31:19.266372+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:19.266372+0000","last_undegraded":"2024-09-22T03:31:19.266372+0000","last_fullsized":"2024-09-22T03:31:19.266372+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":492,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.298195+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:31:19.298195+0000","last_peered":"2024-09-22T03:31:19.298195+0000","last_clean":"2024-09-22T03:31:19.298195+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:19.298195+0000","last_undegraded":"2024-09-22T03:31:19.298195+0000","last_fullsized":"2024-09-22T03:31:19.298195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":496,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.296313+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:31:19.296313+0000","last_peered":"2024-09-22T03:31:19.296313+0000","last_clean":"2024-09-22T03:31:19.296313+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:19.296313+0000","last_undegraded":"2024-09-22T03:31:19.296313+0000","last_fullsized":"2024-09-22T03:31:19.296313+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445435,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62892,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94308948,"statfs":{"total":96636764160,"available":96572362752,"internally_reserved":0,"allocated":2060288,"data_stored":852131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57896,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313944,"statfs":{"total":96636764160,"available":96577478656,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58088,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":56000,"kb_avail":94313752,"statfs":{"total":96636764160,"available":96577282048,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57344000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38116,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2117632,"data_stored":1442480,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:24.821 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T03:31:24.821 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:24.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:24.920+0000 7fc01d82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:24.927+0000 7fc01d82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:25.102 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:25.102 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:25.114 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":851,"stamp":"2024-09-22T03:31:23.484005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":216992,"kb_used_data":8224,"kb_used_omap":0,"kb_used_meta":208704,"kb_avail":377270368,"statfs":{"total":386547056640,"available":386324856832,"internally_reserved":0,"allocated":8421376,"data_stored":5179609,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213712896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.233831"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":459,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268139+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:31:19.268139+0000","last_peered":"2024-09-22T03:31:19.268139+0000","last_clean":"2024-09-22T03:31:19.268139+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:19.268139+0000","last_undegraded":"2024-09-22T03:31:19.268139+0000","last_fullsized":"2024-09-22T03:31:19.268139+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":519,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.278193+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:31:19.278193+0000","last_peered":"2024-09-22T03:31:19.278193+0000","last_clean":"2024-09-22T03:31:19.278193+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:19.278193+0000","last_undegraded":"2024-09-22T03:31:19.278193+0000","last_fullsized":"2024-09-22T03:31:19.278193+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":651,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268560+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:31:19.268560+0000","last_peered":"2024-09-22T03:31:19.268560+0000","last_clean":"2024-09-22T03:31:19.268560+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:19.268560+0000","last_undegraded":"2024-09-22T03:31:19.268560+0000","last_fullsized":"2024-09-22T03:31:19.268560+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":565,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266102+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:31:19.266102+0000","last_peered":"2024-09-22T03:31:19.266102+0000","last_clean":"2024-09-22T03:31:19.266102+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:19.266102+0000","last_undegraded":"2024-09-22T03:31:19.266102+0000","last_fullsized":"2024-09-22T03:31:19.266102+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":636,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.277105+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:31:19.277105+0000","last_peered":"2024-09-22T03:31:19.277105+0000","last_clean":"2024-09-22T03:31:19.277105+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:19.277105+0000","last_undegraded":"2024-09-22T03:31:19.277105+0000","last_fullsized":"2024-09-22T03:31:19.277105+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":417,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.297740+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:31:19.297740+0000","last_peered":"2024-09-22T03:31:19.297740+0000","last_clean":"2024-09-22T03:31:19.297740+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:19.297740+0000","last_undegraded":"2024-09-22T03:31:19.297740+0000","last_fullsized":"2024-09-22T03:31:19.297740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":543,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266372+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:31:19.266372+0000","last_peered":"2024-09-22T03:31:19.266372+0000","last_clean":"2024-09-22T03:31:19.266372+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:19.266372+0000","last_undegraded":"2024-09-22T03:31:19.266372+0000","last_fullsized":"2024-09-22T03:31:19.266372+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":492,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.298195+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:31:19.298195+0000","last_peered":"2024-09-22T03:31:19.298195+0000","last_clean":"2024-09-22T03:31:19.298195+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:19.298195+0000","last_undegraded":"2024-09-22T03:31:19.298195+0000","last_fullsized":"2024-09-22T03:31:19.298195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":496,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.296313+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:31:19.296313+0000","last_peered":"2024-09-22T03:31:19.296313+0000","last_clean":"2024-09-22T03:31:19.296313+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:19.296313+0000","last_undegraded":"2024-09-22T03:31:19.296313+0000","last_fullsized":"2024-09-22T03:31:19.296313+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445435,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62892,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94308948,"statfs":{"total":96636764160,"available":96572362752,"internally_reserved":0,"allocated":2060288,"data_stored":852131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57896,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313944,"statfs":{"total":96636764160,"available":96577478656,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58088,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":56000,"kb_avail":94313752,"statfs":{"total":96636764160,"available":96577282048,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57344000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38116,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2117632,"data_stored":1442480,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:25.115 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:31:25.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:25.216+0000 7fe46d6fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:25.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:25.223+0000 7fe46d6fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:25.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:25.402 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":248,"fsid":"39bf19db-1c34-4685-8c41-ee47ed066388","created":"2024-09-22T03:10:55.010188+0000","modified":"2024-09-22T03:31:19.258843+0000","last_up_change":"2024-09-22T03:30:53.170699+0000","last_in_change":"2024-09-22T03:10:57.091220+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-22T03:11:05.202857+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-22T03:11:05.252894+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e9a0822-2cc0-4936-a416-074895e4a8c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":204,"last_clean_end":228,"up_from":233,"up_thru":237,"down_at":229,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3847282613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":3847282613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3847282613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":3847282613}]},"public_addr":"172.21.15.146:6804/3847282613","cluster_addr":"172.21.15.146:6805/3847282613","heartbeat_back_addr":"172.21.15.146:6812/3847282613","heartbeat_front_addr":"172.21.15.146:6811/3847282613","state":["exists","up"]},{"osd":1,"uuid":"838297aa-1306-4265-aebc-f5a153440ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":159,"last_clean_end":206,"up_from":209,"up_thru":236,"down_at":207,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":2584126373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2584126373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2584126373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":2584126373}]},"public_addr":"172.21.15.146:6801/2584126373","cluster_addr":"172.21.15.146:6800/2584126373","heartbeat_back_addr":"172.21.15.146:6802/2584126373","heartbeat_front_addr":"172.21.15.146:6803/2584126373","state":["exists","up"]},{"osd":2,"uuid":"76e8173e-b2ac-46f1-9dbc-131bb527a587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":178,"last_clean_end":180,"up_from":185,"up_thru":234,"down_at":183,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2090648228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6805","nonce":2090648228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":2090648228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2090648228}]},"public_addr":"172.21.15.148:6806/2090648228","cluster_addr":"172.21.15.148:6805/2090648228","heartbeat_back_addr":"172.21.15.148:6809/2090648228","heartbeat_front_addr":"172.21.15.148:6804/2090648228","state":["exists","up"]},{"osd":3,"uuid":"d32dd872-5134-4314-931d-4de8663e2c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":227,"last_clean_end":228,"up_from":231,"up_thru":238,"down_at":230,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6811","nonce":1614916954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1614916954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6807","nonce":1614916954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1614916954}]},"public_addr":"172.21.15.148:6811/1614916954","cluster_addr":"172.21.15.148:6810/1614916954","heartbeat_back_addr":"172.21.15.148:6807/1614916954","heartbeat_front_addr":"172.21.15.148:6808/1614916954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-22T03:30:20.161661+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"2024-09-22T03:28:32.148741+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"2024-09-22T03:26:48.178527+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"2024-09-22T03:30:21.173762+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:31:26.416 INFO:tasks.ceph:Scrubbing osd.0 2024-09-22T03:31:26.416 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:31:26.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:26.521+0000 7faac1197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:26.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:26.531+0000 7faac1197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:26.543 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:31:26.543 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "" 2024-09-22T03:31:26.543 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:31:26.555 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-22T03:31:26.659 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:26.654+0000 7f0a800d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:26.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:26.662+0000 7f0a800d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:26.839 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 0 to deep-scrub 2024-09-22T03:31:26.857 INFO:tasks.ceph:Scrubbing osd.1 2024-09-22T03:31:26.857 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:31:26.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:26.958+0000 7face1b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:26.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:26.968+0000 7face1b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:26.980 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:31:26.981 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "" 2024-09-22T03:31:26.981 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:31:26.992 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-22T03:31:27.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.091+0000 7faf60397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.099+0000 7faf60397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.281 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 1 to deep-scrub 2024-09-22T03:31:27.297 INFO:tasks.ceph:Scrubbing osd.2 2024-09-22T03:31:27.297 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:31:27.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.398+0000 7f3fc606c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.408+0000 7f3fc606c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.422 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:31:27.422 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "" 2024-09-22T03:31:27.422 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:31:27.433 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-22T03:31:27.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.532+0000 7f714f757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.540+0000 7f714f757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.717 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 2 to deep-scrub 2024-09-22T03:31:27.731 INFO:tasks.ceph:Scrubbing osd.3 2024-09-22T03:31:27.731 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:31:27.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.830+0000 7fc973157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.841+0000 7fc973157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.854 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-22T03:31:27.855 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "" 2024-09-22T03:31:27.855 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-22T03:31:27.866 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-22T03:31:27.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.966+0000 7f675ca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:27.973+0000 7f675ca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:28.150 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 3 to deep-scrub 2024-09-22T03:31:28.164 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:28.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:28.264+0000 7fad0602c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:28.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:28.271+0000 7fad0602c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:28.448 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:28.448 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:28.461 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":853,"stamp":"2024-09-22T03:31:27.484604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":216992,"kb_used_data":8224,"kb_used_omap":0,"kb_used_meta":208704,"kb_avail":377270368,"statfs":{"total":386547056640,"available":386324856832,"internally_reserved":0,"allocated":8421376,"data_stored":5179609,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213712896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.230591"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":459,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268139+0000","last_change":"2024-09-22T03:30:55.354242+0000","last_active":"2024-09-22T03:31:19.268139+0000","last_peered":"2024-09-22T03:31:19.268139+0000","last_clean":"2024-09-22T03:31:19.268139+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:19.268139+0000","last_undegraded":"2024-09-22T03:31:19.268139+0000","last_fullsized":"2024-09-22T03:31:19.268139+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":519,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.278193+0000","last_change":"2024-09-22T03:30:55.192004+0000","last_active":"2024-09-22T03:31:19.278193+0000","last_peered":"2024-09-22T03:31:19.278193+0000","last_clean":"2024-09-22T03:31:19.278193+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:19.278193+0000","last_undegraded":"2024-09-22T03:31:19.278193+0000","last_fullsized":"2024-09-22T03:31:19.278193+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":651,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.268560+0000","last_change":"2024-09-22T03:30:53.218879+0000","last_active":"2024-09-22T03:31:19.268560+0000","last_peered":"2024-09-22T03:31:19.268560+0000","last_clean":"2024-09-22T03:31:19.268560+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:19.268560+0000","last_undegraded":"2024-09-22T03:31:19.268560+0000","last_fullsized":"2024-09-22T03:31:19.268560+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":565,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266102+0000","last_change":"2024-09-22T03:30:55.253164+0000","last_active":"2024-09-22T03:31:19.266102+0000","last_peered":"2024-09-22T03:31:19.266102+0000","last_clean":"2024-09-22T03:31:19.266102+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:19.266102+0000","last_undegraded":"2024-09-22T03:31:19.266102+0000","last_fullsized":"2024-09-22T03:31:19.266102+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":636,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.277105+0000","last_change":"2024-09-22T03:30:54.291654+0000","last_active":"2024-09-22T03:31:19.277105+0000","last_peered":"2024-09-22T03:31:19.277105+0000","last_clean":"2024-09-22T03:31:19.277105+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:19.277105+0000","last_undegraded":"2024-09-22T03:31:19.277105+0000","last_fullsized":"2024-09-22T03:31:19.277105+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":417,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.297740+0000","last_change":"2024-09-22T03:30:55.386923+0000","last_active":"2024-09-22T03:31:19.297740+0000","last_peered":"2024-09-22T03:31:19.297740+0000","last_clean":"2024-09-22T03:31:19.297740+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:19.297740+0000","last_undegraded":"2024-09-22T03:31:19.297740+0000","last_fullsized":"2024-09-22T03:31:19.297740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":543,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.266372+0000","last_change":"2024-09-22T03:30:55.192088+0000","last_active":"2024-09-22T03:31:19.266372+0000","last_peered":"2024-09-22T03:31:19.266372+0000","last_clean":"2024-09-22T03:31:19.266372+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:19.266372+0000","last_undegraded":"2024-09-22T03:31:19.266372+0000","last_fullsized":"2024-09-22T03:31:19.266372+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":492,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.298195+0000","last_change":"2024-09-22T03:30:53.211899+0000","last_active":"2024-09-22T03:31:19.298195+0000","last_peered":"2024-09-22T03:31:19.298195+0000","last_clean":"2024-09-22T03:31:19.298195+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:19.298195+0000","last_undegraded":"2024-09-22T03:31:19.298195+0000","last_fullsized":"2024-09-22T03:31:19.298195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":496,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.296313+0000","last_change":"2024-09-22T03:30:55.220231+0000","last_active":"2024-09-22T03:31:19.296313+0000","last_peered":"2024-09-22T03:31:19.296313+0000","last_clean":"2024-09-22T03:31:19.296313+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:19.296313+0000","last_undegraded":"2024-09-22T03:31:19.296313+0000","last_fullsized":"2024-09-22T03:31:19.296313+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:11:05.253234+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445440,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62892,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94308948,"statfs":{"total":96636764160,"available":96572362752,"internally_reserved":0,"allocated":2060288,"data_stored":852131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57896,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313944,"statfs":{"total":96636764160,"available":96577478656,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58088,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":56000,"kb_avail":94313752,"statfs":{"total":96636764160,"available":96577282048,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57344000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727379998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38116,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2117632,"data_stored":1442480,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:28.463 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-22T03:11:05.253234+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=11, tm_sec=5, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=31, tm_sec=25, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:31:28.464 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-22T03:31:48.466 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:48.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:48.581+0000 7f9d0f397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:48.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:48.587+0000 7f9d0f397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:48.764 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-22T03:31:48.764 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-09-22T03:31:48.777 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":863,"stamp":"2024-09-22T03:31:47.488734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":216992,"kb_used_data":8224,"kb_used_omap":0,"kb_used_meta":208704,"kb_avail":377270368,"statfs":{"total":386547056640,"available":386324856832,"internally_reserved":0,"allocated":8421376,"data_stored":5179609,"data_compressed":89916,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213712896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002657"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":461,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:28.150832+0000","last_change":"2024-09-22T03:31:28.157387+0000","last_active":"2024-09-22T03:31:28.150832+0000","last_peered":"2024-09-22T03:31:28.150832+0000","last_clean":"2024-09-22T03:31:28.150832+0000","last_became_active":"2024-09-22T03:30:54.283187+0000","last_became_peered":"2024-09-22T03:30:54.283187+0000","last_unstale":"2024-09-22T03:31:28.150832+0000","last_undegraded":"2024-09-22T03:31:28.150832+0000","last_fullsized":"2024-09-22T03:31:28.150832+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:31:28.157264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:28.157264+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":521,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.841559+0000","last_change":"2024-09-22T03:31:26.854788+0000","last_active":"2024-09-22T03:31:26.841559+0000","last_peered":"2024-09-22T03:31:26.841559+0000","last_clean":"2024-09-22T03:31:26.841559+0000","last_became_active":"2024-09-22T03:30:54.250166+0000","last_became_peered":"2024-09-22T03:30:54.250166+0000","last_unstale":"2024-09-22T03:31:26.841559+0000","last_undegraded":"2024-09-22T03:31:26.841559+0000","last_fullsized":"2024-09-22T03:31:26.841559+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:31:26.854687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:26.854687+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":653,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:27.717962+0000","last_change":"2024-09-22T03:31:27.728620+0000","last_active":"2024-09-22T03:31:27.717962+0000","last_peered":"2024-09-22T03:31:27.717962+0000","last_clean":"2024-09-22T03:31:27.717962+0000","last_became_active":"2024-09-22T03:30:23.269145+0000","last_became_peered":"2024-09-22T03:30:23.269145+0000","last_unstale":"2024-09-22T03:31:27.717962+0000","last_undegraded":"2024-09-22T03:31:27.717962+0000","last_fullsized":"2024-09-22T03:31:27.717962+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2024-09-22T03:31:27.728432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:27.728432+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":567,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:28.150829+0000","last_change":"2024-09-22T03:31:28.198659+0000","last_active":"2024-09-22T03:31:28.150829+0000","last_peered":"2024-09-22T03:31:28.150829+0000","last_clean":"2024-09-22T03:31:28.150829+0000","last_became_active":"2024-09-22T03:30:54.251361+0000","last_became_peered":"2024-09-22T03:30:54.251361+0000","last_unstale":"2024-09-22T03:31:28.150829+0000","last_undegraded":"2024-09-22T03:31:28.150829+0000","last_fullsized":"2024-09-22T03:31:28.150829+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'1","last_scrub_stamp":"2024-09-22T03:31:28.198534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:28.198534+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"12'97","reported_seq":638,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.840600+0000","last_change":"2024-09-22T03:31:26.897410+0000","last_active":"2024-09-22T03:31:26.840600+0000","last_peered":"2024-09-22T03:31:26.840600+0000","last_clean":"2024-09-22T03:31:26.840600+0000","last_became_active":"2024-09-22T03:30:54.266884+0000","last_became_peered":"2024-09-22T03:30:54.266884+0000","last_unstale":"2024-09-22T03:31:26.840600+0000","last_undegraded":"2024-09-22T03:31:26.840600+0000","last_fullsized":"2024-09-22T03:31:26.840600+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'97","last_scrub_stamp":"2024-09-22T03:31:26.897168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:26.897168+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":419,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:27.282294+0000","last_change":"2024-09-22T03:31:27.289500+0000","last_active":"2024-09-22T03:31:27.282294+0000","last_peered":"2024-09-22T03:31:27.282294+0000","last_clean":"2024-09-22T03:31:27.282294+0000","last_became_active":"2024-09-22T03:30:54.266578+0000","last_became_peered":"2024-09-22T03:30:54.266578+0000","last_unstale":"2024-09-22T03:31:27.282294+0000","last_undegraded":"2024-09-22T03:31:27.282294+0000","last_fullsized":"2024-09-22T03:31:27.282294+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:31:27.289419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:27.289419+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":545,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:27.717889+0000","last_change":"2024-09-22T03:31:27.725522+0000","last_active":"2024-09-22T03:31:27.717889+0000","last_peered":"2024-09-22T03:31:27.717889+0000","last_clean":"2024-09-22T03:31:27.717889+0000","last_became_active":"2024-09-22T03:30:54.259814+0000","last_became_peered":"2024-09-22T03:30:54.259814+0000","last_unstale":"2024-09-22T03:31:27.717889+0000","last_undegraded":"2024-09-22T03:31:27.717889+0000","last_fullsized":"2024-09-22T03:31:27.717889+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:31:27.725270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:27.725270+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":494,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:27.282676+0000","last_change":"2024-09-22T03:31:27.288690+0000","last_active":"2024-09-22T03:31:27.282676+0000","last_peered":"2024-09-22T03:31:27.282676+0000","last_clean":"2024-09-22T03:31:27.282676+0000","last_became_active":"2024-09-22T03:30:23.268687+0000","last_became_peered":"2024-09-22T03:30:23.268687+0000","last_unstale":"2024-09-22T03:31:27.282676+0000","last_undegraded":"2024-09-22T03:31:27.282676+0000","last_fullsized":"2024-09-22T03:31:27.282676+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:31:27.288601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:27.288601+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":498,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-22T03:31:27.282292+0000","last_change":"2024-09-22T03:31:27.307941+0000","last_active":"2024-09-22T03:31:27.282292+0000","last_peered":"2024-09-22T03:31:27.282292+0000","last_clean":"2024-09-22T03:31:27.282292+0000","last_became_active":"2024-09-22T03:30:54.238967+0000","last_became_peered":"2024-09-22T03:30:54.238967+0000","last_unstale":"2024-09-22T03:31:27.282292+0000","last_undegraded":"2024-09-22T03:31:27.282292+0000","last_fullsized":"2024-09-22T03:31:27.282292+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:31:27.307853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:31:27.307853+0000","last_clean_scrub_stamp":"2024-09-22T03:11:05.253234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":59944,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":231,"seq":992137445460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62892,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94308948,"statfs":{"total":96636764160,"available":96572362752,"internally_reserved":0,"allocated":2060288,"data_stored":852131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":185,"seq":794568950040,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57896,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":55808,"kb_avail":94313944,"statfs":{"total":96636764160,"available":96577478656,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57147392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":209,"seq":897648165048,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58088,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":56000,"kb_avail":94313752,"statfs":{"total":96636764160,"available":96577282048,"internally_reserved":0,"allocated":2121728,"data_stored":1442499,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":57344000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":233,"seq":1000727380018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38116,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2117632,"data_stored":1442480,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29972,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:48.778 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-22T03:31:48.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:48.875+0000 7fabaed57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:48.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-22T03:31:48.885+0000 7fabaed57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:49.080 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-22T03:31:49.081 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-22T03:31:49.081 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T03:31:49.081 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:31:53.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-22T03:31:55.183 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T03:31:55.183 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:31:55.183 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:31:58.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:31:58.568 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-22T03:32:01.284 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:32:01.285 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:32:01.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:32:03.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:32:03.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:32:07.386 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:32:07.386 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T03:32:07.387 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:32:09.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T03:32:09.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:32:13.489 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T03:32:13.489 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-22T03:32:13.489 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-22T03:32:13.489 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:32:13.521 INFO:tasks.ceph.mgr.x:Stopped 2024-09-22T03:32:13.521 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-22T03:32:13.522 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-22T03:32:13.522 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:32:13.538 INFO:tasks.ceph.mon.a:Stopped 2024-09-22T03:32:13.538 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-22T03:32:13.541 DEBUG:teuthology.orchestra.run.smithi146:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-22T03:32:13.619 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-22T03:32:13.620 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-22T03:32:13.813 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-22T03:32:13.813 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-22T03:32:13.916 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-22T03:32:13.916 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-22T03:32:13.982 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-22T03:32:13.982 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-22T03:32:14.053 INFO:tasks.ceph:Archiving mon data... 2024-09-22T03:32:14.054 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732/data/mon.a.tgz 2024-09-22T03:32:14.054 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-22T03:32:14.073 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.RajpJE4f7f 2024-09-22T03:32:14.073 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RajpJE4f7f 2024-09-22T03:32:14.073 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T03:32:14.210 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.RajpJE4f7f 2024-09-22T03:32:14.295 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.RajpJE4f7f is 319KB 2024-09-22T03:32:14.391 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.RajpJE4f7f 2024-09-22T03:32:14.406 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-22T03:32:14.406 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T03:32:14.447 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T03:32:14.525 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T03:32:14.542 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T03:32:14.543 INFO:tasks.ceph:Archiving crash dumps... 2024-09-22T03:32:14.544 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732/remote/smithi146/crash 2024-09-22T03:32:14.545 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T03:32:14.591 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732/remote/smithi148/crash 2024-09-22T03:32:14.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T03:32:14.665 INFO:tasks.ceph:Compressing logs... 2024-09-22T03:32:14.665 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T03:32:14.667 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T03:32:14.690 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-22T03:32:14.690 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55933.log 2024-09-22T03:32:14.691 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38674.log 2024-09-22T03:32:14.691 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57507.log 2024-09-22T03:32:14.691 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55933.log.gz 2024-09-22T03:32:14.691 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37487.log 2024-09-22T03:32:14.691 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38674.log.gz 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28510.log 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28419.log 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28550.log 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.28510.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28510.log.gz 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.28419.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28419.log.gz 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.28550.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28550.log.gz 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28470.log 2024-09-22T03:32:14.701 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-22T03:32:14.702 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.28470.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28470.log.gz 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54460.log 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57507.log.gz 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37487.log.gz 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29596.log 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54460.log.gz 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55612.log 2024-09-22T03:32:14.783 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-09-22T03:32:14.784 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28652.log 2024-09-22T03:32:14.784 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29596.log.gz 2024-09-22T03:32:14.784 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52585.log 2024-09-22T03:32:14.784 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55612.log.gz 2024-09-22T03:32:14.785 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.28652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50043.log 2024-09-22T03:32:14.785 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28652.log.gz 2024-09-22T03:32:14.785 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51920.log 2024-09-22T03:32:14.785 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52585.log.gz 2024-09-22T03:32:14.785 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30698.log 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50043.log.gz 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51920.log.gz 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35529.log 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30698.log.gz 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53188.log 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47452.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29512.log 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35529.log.gz 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52714.log 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53188.log.gz 2024-09-22T03:32:14.786 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37318.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29512.log.gz 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56817.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52714.log.gz 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50137.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37318.log.gz 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40688.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56817.log.gz 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31419.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50137.log.gz 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45326.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40688.log.gz 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34986.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.31419.log.gz 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28908.log 2024-09-22T03:32:14.787 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45326.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56355.log 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34986.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.28908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35916.log 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28908.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57028.log 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56355.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56286.log 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35916.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50940.log 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57028.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36898.log 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56286.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37178.log 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50940.log.gz 2024-09-22T03:32:14.788 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40925.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36898.log.gz 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51297.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37178.log.gz 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49059.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40925.log.gz 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52754.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51297.log.gz 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43918.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.49059.log.gz 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54509.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52754.log.gz 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46465.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43918.log.gz 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41176.log 2024-09-22T03:32:14.789 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.54509.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30627.log 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46465.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55389.log 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41176.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57784.log 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30627.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55017.log 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55389.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43304.log 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57784.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41385.log 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55017.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53719.log 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43304.log.gz 2024-09-22T03:32:14.790 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31337.log 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41385.log.gz 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40473.log 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53719.log.gz 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36043.log 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31337.log.gz 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46556.log 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40473.log.gz 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45557.log 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.36043.log.gz 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50344.log 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46556.log.gz 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30565.log 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45557.log.gz 2024-09-22T03:32:14.791 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38576.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50344.log.gz 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49188.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30565.log.gz 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57337.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38576.log.gz 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43482.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.49188.log.gz 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56136.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 48.8% -- replaced with /var/log/ceph/ceph-client.admin.57337.log.gz 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50891.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.43482.log.gz 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57112.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56136.log.gz 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56860.log 2024-09-22T03:32:14.792 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50891.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39516.log 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57112.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29826.log 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56860.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56860.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47669.log 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39516.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43255.log 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29826.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54823.log 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47669.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34538.log 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43255.log: 49.6%/var/log/ceph/ceph-client.admin.54823.log: -- replaced with /var/log/ceph/ceph-client.admin.43255.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31640.log 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54823.log.gz 2024-09-22T03:32:14.793 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56684.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34538.log.gz 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33691.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.31640.log.gz 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34831.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56684.log.gz 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33691.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.33691.log.gz 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29996.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.34831.log.gz 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27352.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.0.29996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58120.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.tmp-client.admin.27352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36541.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27352.log.gz 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31044.log 2024-09-22T03:32:14.794 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58120.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36541.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44879.log 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31044.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48730.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48443.log 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45508.log 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44879.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48443.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48443.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57070.log 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32795.log 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45508.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45508.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57070.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57070.log.gz 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32970.log 2024-09-22T03:32:14.795 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32795.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32795.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33243.log 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32970.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49268.log 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55347.log 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33243.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53419.log 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49268.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56593.log 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54725.log 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53419.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53419.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56593.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56593.log.gz 2024-09-22T03:32:14.796 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52171.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32123.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54725.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54725.log.gz 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52171.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.52171.log.gz 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56986.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57506.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32123.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32123.log.gz 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56986.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56986.log.gz 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31990.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54551.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57506.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57506.log.gz 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31990.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31990.log.gz 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47900.log 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54551.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54551.log.gz 2024-09-22T03:32:14.797 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44058.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.51563.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42826.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47900.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47900.log.gz 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31490.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36163.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42826.log.gz 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53488.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.31490.log.gz 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49836.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36163.log.gz 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34006.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53488.log.gz 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29344.log 2024-09-22T03:32:14.798 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49836.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42374.log 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.34006.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47186.log 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29344.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48961.log 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42374.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45835.log 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47186.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48347.log 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48961.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30215.log 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45835.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44461.log 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48347.log.gz 2024-09-22T03:32:14.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50391.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30215.log.gz 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44461.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44461.log.gz 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50391.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50391.log.gz 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49996.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38128.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50217.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49996.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49996.log.gz 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33152.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57196.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38128.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38128.log.gz 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33152.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33152.log.gz 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44187.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38306.log 2024-09-22T03:32:14.800 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57196.log: 49.6%/var/log/ceph/ceph-client.admin.44187.log: -- replaced with /var/log/ceph/ceph-client.admin.57196.log.gz 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 49.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.44187.log.gz -- 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.admin.51072.log 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50257.log 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38306.log.gz 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41527.log 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51072.log.gz 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56544.log 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50257.log.gz 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53586.log 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41527.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41527.log.gz 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56544.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56544.log.gz 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53586.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53586.log.gz 2024-09-22T03:32:14.801 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45766.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz -5 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38533.log 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35618.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38533.log: gzip 48.0% -- replaced with /var/log/ceph/ceph-client.admin.38533.log.gz -5 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49916.log 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52985.log 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44227.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49916.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49916.log.gz 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43522.log 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29134.log 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52985.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52985.log.gz 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43522.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43522.log.gz 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35173.log 2024-09-22T03:32:14.802 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42617.log 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29134.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29134.log.gz 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35173.log: gzip 50.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.35173.log.gz --verbose 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr: -- /var/log/ceph/ceph-client.admin.30747.log 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33600.log 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42617.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.42617.log.gz 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29234.log 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30747.log.gz 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43374.log 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33600.log.gz 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32032.log 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29234.log.gz 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52130.log 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43374.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43374.log.gz 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32032.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32032.log.gz 2024-09-22T03:32:14.803 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37447.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45459.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52130.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52130.log.gz 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37447.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37447.log.gz 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45459.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45459.log.gz 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36849.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51738.log: /var/log/ceph/ceph-client.admin.43660.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39121.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36849.log.gz 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47494.log 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43660.log.gz 2024-09-22T03:32:14.804 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43611.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39121.log.gz 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45984.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47494.log.gz 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43611.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.47940.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43611.log.gz 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48150.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45984.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45984.log.gz 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47940.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.47940.log.gz 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31856.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41019.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.0.48150.log: /var/log/ceph/ceph-client.admin.31856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54870.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.31856.log.gz 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:gzip/var/log/ceph/ceph-client.admin.41019.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51381.log 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41019.log.gz 2024-09-22T03:32:14.805 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51339.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54870.log.gz 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55206.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51381.log.gz 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51696.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51339.log.gz 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52022.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55206.log.gz 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55478.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51696.log.gz 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31268.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52022.log.gz 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55478.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.55478.log.gz 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57154.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43434.log 2024-09-22T03:32:14.806 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31268.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31268.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57154.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57154.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr: -- /var/log/ceph/ceph-client.admin.34230.log 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43434.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43434.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34230.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34230.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr: -- /var/log/ceph/ceph-client.admin.52310.log 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39208.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54593.log 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52310.log: gzip -5 --verbose 50.4% -- replaced with /var/log/ceph/ceph-client.admin.52310.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr: -- /var/log/ceph/ceph-client.admin.34902.log 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54593.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54593.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36401.log 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34902.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34902.log.gz 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.admin.44412.log 2024-09-22T03:32:14.807 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36401.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36401.log.gz 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45717.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44412.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44412.log.gz 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49956.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45717.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45717.log.gz 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55066.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54245.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49956.log.gz 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47359.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55066.log.gz 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40593.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54245.log.gz 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58372.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47359.log.gz 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-22T03:32:14.808 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40593.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39681.log 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58372.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54635.log 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39681.log: gzip -5 50.0% --verbose -- /var/log/ceph/ceph-client.admin.30348.log 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39681.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55826.log 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54635.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36003.log 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.30348.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40222.log 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55826.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38266.log 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36003.log.gz 2024-09-22T03:32:14.809 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40222.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40222.log.gzgzip 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54373.log 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38266.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38266.log.gz 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32522.log 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54373.log: gzip -5 --verbose 50.4% -- /var/log/ceph/ceph-client.admin.28991.log -- replaced with /var/log/ceph/ceph-client.admin.54373.log.gz 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32522.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32522.log.gz 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.28991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58246.log 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr: 67.2% -- replaced with /var/log/ceph/ceph-client.admin.28991.log.gz 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48870.log: /var/log/ceph/ceph-client.admin.29050.log 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr:gzip/var/log/ceph/ceph-client.admin.58246.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47627.log 2024-09-22T03:32:14.810 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58246.log.gz/var/log/ceph/ceph-client.admin.29050.log: 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.29050.log.gz 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37625.log 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42924.log 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47627.log.gz 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51605.log 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37625.log.gz 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56404.log 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42924.log.gz 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51164.log 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51605.log.gz 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56404.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.44510.log 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56404.log.gz 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44559.log 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51164.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51164.log.gz 2024-09-22T03:32:14.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42049.log 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44510.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44559.log: gzip -5 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44559.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47578.log 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42049.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:gzip/var/log/ceph/ceph-client.admin.48031.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55572.log 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47578.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47578.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47991.log 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56775.log 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55572.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.55572.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47991.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.38997.log 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47991.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56775.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56775.log.gz 2024-09-22T03:32:14.812 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49317.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32396.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38997.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38997.log.gz 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49317.log: gzip -5 49.6% --verbose -- /var/log/ceph/ceph-client.admin.57338.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49317.log.gz 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32072.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.32396.log.gz 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41485.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57338.log: 48.8% -- replaced with /var/log/ceph/ceph-client.admin.57338.log.gz 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32072.log: gzip -5 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32072.log.gz 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58498.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41485.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41485.log.gz 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49597.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58498.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph.audit.log 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58498.log.gz 2024-09-22T03:32:14.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49597.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49597.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58330.log 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46647.log 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.58330.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58330.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.admin.33467.log 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46647.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46647.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32347.log 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33467.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33467.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48681.log 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32347.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32347.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48681.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48681.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.admin.31948.log 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52843.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-22T03:32:14.814 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45597.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29554.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31948.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31948.log.gz 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45597.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45597.log.gz -5 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36590.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56002.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29554.log: 50.0%/var/log/ceph/ceph-client.admin.36590.log: -- replaced with /var/log/ceph/ceph-client.admin.29554.log.gz 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36590.log.gz 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48207.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55732.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56002.log.gz 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48207.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48207.log.gz 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:gzip/var/log/ceph/ceph-client.admin.55732.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49876.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55732.log.gz 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30430.log 2024-09-22T03:32:14.815 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49876.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49876.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.admin.38435.log 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30430.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.30430.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40386.log 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38435.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45417.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.admin.56635.log 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40386.log: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.40386.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45134.log 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56635.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56635.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45134.log: gzip -5 --verbose -- 50.4% -- replaced with /var/log/ceph/ceph-client.admin.45134.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.0.35946.log 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53936.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2024-09-22T03:32:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48779.log 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.0.35946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36306.log 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29404.log 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48779.log.gz 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36306.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36306.log.gz 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53537.log 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35480.log/var/log/ceph/ceph-client.admin.29404.log: 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53537.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29404.log.gz 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53537.log.gz 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35480.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35480.log.gz 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33866.log 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40553.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48590.log 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33866.log: gzip -5 --verbose -- 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33866.log.gz 2024-09-22T03:32:14.817 INFO:teuthology.orchestra.run.smithi146.stderr: /var/log/ceph/ceph-client.admin.39882.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48590.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48590.log.gz 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35035.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40878.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39882.log: 49.6%/var/log/ceph/ceph-client.admin.35035.log: -- replaced with /var/log/ceph/ceph-client.admin.39882.log.gz 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35035.log.gz 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28964.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40878.log: gzip 50.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.29722.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40878.log.gz 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.28964.log: gzip 49.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.51472.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28964.log.gz 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47851.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29722.log.gz 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41434.log 2024-09-22T03:32:14.818 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51472.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51472.log.gz 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40433.log 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47851.log: /var/log/ceph/ceph-client.admin.41434.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47851.log.gzgzip 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46689.log 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41434.log.gz 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39641.log 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40433.log: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.40433.log.gz 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49366.log 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46689.log: /var/log/ceph/ceph-client.admin.39641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35271.log 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46689.log.gz 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39641.log.gz 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45897.log 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49366.log.gz/var/log/ceph/ceph-client.admin.35271.log: 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35271.log.gz 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr: -- /var/log/ceph/ceph-client.admin.46347.log 2024-09-22T03:32:14.819 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45897.log: 50.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28417.log 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45897.log.gz 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29638.log 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46347.log.gz 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.28417.log: /var/log/ceph/ceph-client.admin.54968.log 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28417.log.gz/var/log/ceph/ceph-client.admin.29638.log: 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40831.log 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29638.log.gz 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53761.log 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54968.log.gz 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50797.log/var/log/ceph/ceph-client.admin.40831.log: 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40831.log.gz/var/log/ceph/ceph-client.admin.53761.log: 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53761.log.gz 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42560.log 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50797.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.50797.log.gz 2024-09-22T03:32:14.820 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34631.log 2024-09-22T03:32:14.821 INFO:teuthology.orchestra.run.smithi146.stderr:gzip/var/log/ceph/ceph-client.0.42560.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52634.log 2024-09-22T03:32:14.821 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34811.log 2024-09-22T03:32:14.821 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34631.log.gz 2024-09-22T03:32:14.821 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33334.log 2024-09-22T03:32:14.821 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34811.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52634.log.gz 2024-09-22T03:32:14.821 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34811.log.gz 2024-09-22T03:32:14.839 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.3.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-22T03:32:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43740.log 2024-09-22T03:32:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58036.log 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.33334.log.gz 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35311.log 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43740.log.gz 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37269.log 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6%/var/log/ceph/ceph-client.admin.35311.log: -- replaced with /var/log/ceph/ceph-client.admin.58036.log.gz 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35311.log.gz 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33558.log 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34188.log 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37269.log.gz 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39161.log 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34188.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33558.log.gz 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34188.log.gz 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32480.log 2024-09-22T03:32:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39161.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34363.log 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39161.log.gz 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37919.log 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32480.log: /var/log/ceph/ceph-client.admin.34363.log: 50.0% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32480.log.gz 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34363.log.gz 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51121.log 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37919.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31723.log 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37919.log.gz 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45943.log 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51121.log: /var/log/ceph/ceph-client.admin.31723.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51121.log.gz 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31723.log.gz 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48919.log 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55436.log 2024-09-22T03:32:14.980 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45943.log.gz 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57616.log 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48919.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48919.log.gz 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55436.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55436.log.gz 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35126.log 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57616.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57616.log.gz 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50090.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32298.log 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54020.log 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35126.log: /var/log/ceph/ceph-client.admin.32298.log: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35126.log.gz 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32298.log.gz 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47235.log 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54020.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54020.log.gz 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-09-22T03:32:14.981 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53669.log 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47235.log.gz 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31091.log 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31091.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.53669.log.gz 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31091.log.gz 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37527.log 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39078.log 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48298.log 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39078.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37527.log.gz 2024-09-22T03:32:14.982 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39078.log.gz 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43780.log 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48298.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36940.log 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48298.log.gz 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51829.log 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43780.log: /var/log/ceph/ceph-client.admin.36940.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43780.log.gz 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36940.log.gz 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40737.log 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51829.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51829.log.gz 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40029.log 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54776.log 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40737.log.gz 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34678.log 2024-09-22T03:32:14.983 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40029.log.gz/var/log/ceph/ceph-client.admin.54776.log: 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.54776.log.gz 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30833.log 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42530.log 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34678.log.gz 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33019.log 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30833.log: /var/log/ceph/ceph-client.admin.42530.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30833.log.gz 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42530.log.gz 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56902.log 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45228.log 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33019.log.gz 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41625.log 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56902.log: /var/log/ceph/ceph-client.admin.45228.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56902.log.gz 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.45228.log.gz 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57844.log 2024-09-22T03:32:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35351.log 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.41625.log.gz 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57844.log: /var/log/ceph/ceph-client.admin.35351.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57844.log.gz 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35351.log.gz 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39474.log 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32662.log 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52269.log 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32662.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39474.log.gz 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32662.log.gz 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33110.log 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42746.log 2024-09-22T03:32:14.985 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52269.log.gz 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30605.log 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33110.log: /var/log/ceph/ceph-client.admin.42746.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33110.log.gz 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42746.log.gz 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42974.log 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44775.log 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30605.log.gz 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52536.log 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42974.log.gz 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43072.log 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44775.log.gz 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52536.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.52536.log.gz 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53894.log 2024-09-22T03:32:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5/var/log/ceph/ceph-client.admin.43072.log: --verbose -- /var/log/ceph/ceph-client.admin.44009.log 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43072.log.gz 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53894.log.gz 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50750.log 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44009.log.gz/var/log/ceph/ceph-client.admin.37218.log: 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53802.log 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32746.log 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50750.log.gz 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49546.log 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53802.log.gz 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39561.log 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.32746.log.gz/var/log/ceph/ceph-client.admin.49546.log: 2024-09-22T03:32:14.987 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49546.log.gz 2024-09-22T03:32:14.988 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51871.log 2024-09-22T03:32:14.988 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30873.log 2024-09-22T03:32:14.989 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39561.log.gz 2024-09-22T03:32:14.989 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36989.log 2024-09-22T03:32:14.989 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30873.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51871.log.gz 2024-09-22T03:32:14.989 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30873.log.gz 2024-09-22T03:32:14.989 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2024-09-22T03:32:14.989 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44993.log 2024-09-22T03:32:14.989 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36989.log.gz 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31906.log 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44993.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44993.log.gz 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52674.log 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31906.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35222.log 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31906.log.gz 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34496.log 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52674.log: /var/log/ceph/ceph-client.admin.35222.log: 49.6% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52674.log.gz 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35222.log.gz 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45367.log 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45815.log 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34496.log.gz 2024-09-22T03:32:14.990 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51423.log 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45367.log.gz 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39037.log 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6%/var/log/ceph/ceph-client.admin.51423.log: -- replaced with /var/log/ceph/ceph-client.admin.45815.log.gz 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51423.log.gz 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35055.log 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47277.log 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39037.log.gz 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48396.log 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47277.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35055.log.gz 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47277.log.gz 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31773.log 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30390.log 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48396.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.48396.log.gz 2024-09-22T03:32:14.991 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31773.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.31773.log.gz 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30789.log 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40513.log 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30390.log.gz 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38383.log 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40513.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30789.log.gz 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40513.log.gz 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46514.log 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38383.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39947.log 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38383.log.gz 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46514.log: /var/log/ceph/ceph-client.admin.39947.log: 50.4% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46514.log.gz 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39947.log.gz 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28496.log 2024-09-22T03:32:14.992 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34944.log 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.28496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55525.log 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28496.log.gz 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31133.log 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6%/var/log/ceph/ceph-client.admin.55525.log: -- replaced with /var/log/ceph/ceph-client.admin.34944.log.gz 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55525.log.gz 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53978.log 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31133.log.gz 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58204.log 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53978.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53978.log.gz 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35874.log 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58204.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.58204.log.gz 2024-09-22T03:32:14.993 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38226.log 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35874.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35874.log.gz 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38716.log 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38226.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38226.log.gz 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42875.log 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38716.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42875.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42875.log.gz 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57845.log 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42786.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52775.log 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57845.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.57845.log.gz 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31682.log 2024-09-22T03:32:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52775.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52775.log.gz 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36670.log 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31682.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31682.log.gz 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57448.log 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36670.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36670.log.gz 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39773.log 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57448.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57448.log.gz 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51492.log 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39773.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39773.log.gz 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55779.log 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51492.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51492.log.gz 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29236.log 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55779.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55779.log.gz 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36740.log 2024-09-22T03:32:14.995 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29236.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29236.log.gz 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38048.log 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36740.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36740.log.gz 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45181.log 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38048.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38048.log.gz 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32844.log 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45181.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45181.log.gz 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41980.log 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32844.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32844.log.gz 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55652.log 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41980.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41980.log.gz 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53303.log 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55652.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55652.log.gz 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33734.log 2024-09-22T03:32:14.996 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53303.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53303.log.gz 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38765.log 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33734.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33734.log.gz 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35698.log 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38765.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38765.log.gz 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56726.log 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35698.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35698.log.gz 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54288.log 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56726.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56726.log.gz 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31573.log 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54288.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54288.log.gz 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31573.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31573.log.gz 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34720.log 2024-09-22T03:32:14.997 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47718.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57636.log 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34720.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34720.log.gz 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44100.log 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57636.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57636.log.gz 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43164.log 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44100.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44100.log.gz 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49789.log 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43164.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43164.log.gz 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39255.log 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49789.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49789.log.gz 2024-09-22T03:32:14.998 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40076.log 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39255.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39255.log.gz 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29905.log 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40076.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40076.log.gz 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40273.log 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29905.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29905.log.gz 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47410.log 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40273.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40273.log.gz 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53370.log 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47410.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47410.log.gz 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36800.log 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53370.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53370.log.gz 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55108.log 2024-09-22T03:32:14.999 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36800.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36800.log.gz 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48821.log 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55108.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55108.log.gz 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56234.log 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48821.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48821.log.gz 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51031.log 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56234.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56234.log.gz 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42285.log 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51031.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51031.log.gz 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29092.log 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42285.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33418.log 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29092.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29092.log.gz 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51647.log 2024-09-22T03:32:15.000 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33418.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.33418.log.gz 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51647.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51647.log.gz 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41576.log 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43562.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38336.log 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41576.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41576.log.gz 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38336.log: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.38336.log.gz 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51962.log 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37968.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41305.log 2024-09-22T03:32:15.001 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51962.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51962.log.gz 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36352.log 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41305.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48078.log 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36352.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36352.log.gz 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38008.log 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48078.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48078.log.gz 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56086.log 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38008.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38008.log.gz 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35780.log 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56086.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56086.log.gz 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55157.log 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35780.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35780.log.gz 2024-09-22T03:32:15.002 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29680.log 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55157.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55157.log.gz 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38954.log 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29680.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.29680.log.gz 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32571.log 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38954.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38954.log.gz 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50177.log 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32571.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32571.log.gz 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55253.log 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50177.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50177.log.gz 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30523.log 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55253.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55253.log.gz 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30523.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30523.log.gz 2024-09-22T03:32:15.003 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34048.log 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43213.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33194.log 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34048.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.34048.log.gz 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30997.log 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33194.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33194.log.gz 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37367.log 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30997.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30997.log.gz 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58162.log 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37367.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37367.log.gz 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32256.log 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58162.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.58162.log.gz 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41345.log 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32256.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32256.log.gz 2024-09-22T03:32:15.004 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30481.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41345.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41345.log.gz 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53628.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30481.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.30481.log.gz 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57678.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53628.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53628.log.gz 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30914.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57678.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57678.log.gz 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54154.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30914.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30914.log.gz 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36492.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54154.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54154.log.gz 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49148.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36492.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36492.log.gz 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42488.log 2024-09-22T03:32:15.005 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49148.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49148.log.gz 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36083.log 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42488.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42488.log.gz 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36083.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36083.log.gz 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49646.log 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40339.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gz 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35578.log 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49646.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49646.log.gz 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33642.log 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35578.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35578.log.gz 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47319.log 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33642.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.33642.log.gz 2024-09-22T03:32:15.006 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47319.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47319.log.gz 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45087.log 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46024.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47049.log 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45087.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45087.log.gz 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47049.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47049.log.gz 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29235.log 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.log: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41872.log 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29235.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29235.log.gz 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56453.log 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41872.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41872.log.gz 2024-09-22T03:32:15.007 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45637.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56453.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56453.log.gz 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41265.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45637.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45637.log.gz 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34587.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41265.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41265.log.gz 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50656.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34587.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34587.log.gz 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52893.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50656.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.50656.log.gz 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57994.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52893.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52893.log.gz 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35431.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57994.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.57994.log.gz 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33964.log 2024-09-22T03:32:15.008 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35431.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35431.log.gz 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52942.log 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33964.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33964.log.gz 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58540.log 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52942.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52942.log.gz 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32928.log 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58540.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58540.log.gz 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58078.log 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32928.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32928.log.gz 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57508.log 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58078.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58078.log.gz 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38484.log 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57508.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57508.log.gz 2024-09-22T03:32:15.009 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46730.log 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38484.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38484.log.gz 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49506.log 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46730.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46730.log.gz 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49506.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49506.log.gz 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58582.log 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33915.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54685.log 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58582.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58582.log.gz 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41127.log 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54685.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54685.log.gz 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31815.log 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41127.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41127.log.gz 2024-09-22T03:32:15.010 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57677.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31815.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31815.log.gz 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32214.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57677.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57677.log.gz 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50297.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32214.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32214.log.gz 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38856.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50297.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50297.log.gz 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46864.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38856.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38856.log.gz 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40784.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46864.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.46864.log.gz 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36212.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40784.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40784.log.gz 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29904.log 2024-09-22T03:32:15.011 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36212.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36212.log.gz 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51758.log 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29904.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29904.log.gz 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35658.log 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51758.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51758.log.gz 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48120.log 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35658.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35658.log.gz 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47802.log 2024-09-22T03:32:15.012 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48120.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48120.log.gz 2024-09-22T03:32:15.035 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-09-22T03:32:15.035 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47802.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47802.log.gz 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58288.log 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49010.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57340.log 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58288.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58288.log.gz 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48256.log 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57340.log: 48.8% -- replaced with /var/log/ceph/ceph-client.admin.57340.log.gz 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35738.log 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48256.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48256.log.gz 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34407.log 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35738.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35738.log.gz 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58665.log 2024-09-22T03:32:15.036 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34407.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34407.log.gz 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39601.log 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58665.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58665.log.gz 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46780.log 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39601.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39601.log.gz 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37407.log 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46780.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46780.log.gz 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37407.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37407.log.gz 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58456.log 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48541.log: 47.2% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40640.log 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58456.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58456.log.gz 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32438.log 2024-09-22T03:32:15.037 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40640.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40640.log.gz 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36692.log 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32438.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32438.log.gz 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42706.log 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36692.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36692.log.gz 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46122.log 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42706.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42706.log.gz 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50703.log 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46122.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46122.log.gz 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49102.log 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50703.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50703.log.gz 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37080.log 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49102.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49102.log.gz 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44608.log 2024-09-22T03:32:15.038 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37080.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37080.log.gz 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44316.log 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44608.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44608.log.gz 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50989.log 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44316.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44316.log.gz 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54330.log 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50989.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50989.log.gz 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39727.log 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54330.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54330.log.gz 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41225.log 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.39727.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39727.log.gz 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54403.log 2024-09-22T03:32:15.039 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41225.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41225.log.gz 2024-09-22T03:32:15.040 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43023.log 2024-09-22T03:32:15.040 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.0.54403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52220.log 2024-09-22T03:32:15.040 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43023.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43023.log.gz 2024-09-22T03:32:15.040 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58625.log 2024-09-22T03:32:15.040 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52220.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52220.log.gz 2024-09-22T03:32:15.044 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32886.log 2024-09-22T03:32:15.044 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58625.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58625.log.gz 2024-09-22T03:32:15.054 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34272.log 2024-09-22T03:32:15.054 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32886.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32886.log.gz 2024-09-22T03:32:15.070 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33068.log 2024-09-22T03:32:15.070 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34272.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34272.log.gz 2024-09-22T03:32:15.083 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34454.log 2024-09-22T03:32:15.083 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33068.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33068.log.gz 2024-09-22T03:32:15.094 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2024-09-22T03:32:15.094 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34454.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34454.log.gz 2024-09-22T03:32:15.103 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57339.log 2024-09-22T03:32:15.103 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38814.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gz 2024-09-22T03:32:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33376.log 2024-09-22T03:32:15.121 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57339.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.57339.log.gz 2024-09-22T03:32:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37576.log 2024-09-22T03:32:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33376.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33376.log.gz 2024-09-22T03:32:15.143 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52083.log 2024-09-22T03:32:15.143 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37576.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37576.log.gz 2024-09-22T03:32:15.154 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42332.log 2024-09-22T03:32:15.155 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52083.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52083.log.gz 2024-09-22T03:32:15.171 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54919.log 2024-09-22T03:32:15.172 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42332.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42332.log.gz 2024-09-22T03:32:15.181 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48635.log 2024-09-22T03:32:15.182 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54919.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54919.log.gz 2024-09-22T03:32:15.190 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43114.log 2024-09-22T03:32:15.199 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.48635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55692.log 2024-09-22T03:32:15.203 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43114.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48635.log.gz 2024-09-22T03:32:15.204 INFO:teuthology.orchestra.run.smithi146.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43114.log.gz 2024-09-22T03:32:15.208 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33516.log 2024-09-22T03:32:15.217 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54205.log 2024-09-22T03:32:15.222 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33516.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55692.log.gz 2024-09-22T03:32:15.226 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57238.log 2024-09-22T03:32:15.230 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54205.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33516.log.gz 2024-09-22T03:32:15.235 INFO:teuthology.orchestra.run.smithi146.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.54205.log.gz 2024-09-22T03:32:15.239 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43353.log 2024-09-22T03:32:15.239 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57238.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57238.log.gz 2024-09-22T03:32:15.248 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42394.log 2024-09-22T03:32:15.258 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29233.log 2024-09-22T03:32:15.259 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42394.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42394.log.gz 2024-09-22T03:32:15.261 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43353.log.gz 2024-09-22T03:32:15.266 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56495.log 2024-09-22T03:32:15.275 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30957.log 2024-09-22T03:32:15.275 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56495.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56495.log.gz 2024-09-22T03:32:15.279 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29233.log.gz 2024-09-22T03:32:15.291 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49741.log 2024-09-22T03:32:15.292 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30957.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30957.log.gz 2024-09-22T03:32:15.303 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51227.log 2024-09-22T03:32:15.304 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49741.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49741.log.gz 2024-09-22T03:32:15.312 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-09-22T03:32:15.316 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.51227.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.51227.log.gz 2024-09-22T03:32:15.321 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38905.log 2024-09-22T03:32:15.334 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36123.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-09-22T03:32:15.337 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-09-22T03:32:15.338 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38905.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38905.log.gz 2024-09-22T03:32:15.346 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33782.log 2024-09-22T03:32:15.356 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45040.log 2024-09-22T03:32:15.356 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33782.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33782.log.gz 2024-09-22T03:32:15.359 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz 2024-09-22T03:32:15.364 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47534.log 2024-09-22T03:32:15.377 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45040.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45040.log.gz 2024-09-22T03:32:15.383 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29402.log 2024-09-22T03:32:15.383 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47534.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.47534.log.gz 2024-09-22T03:32:15.394 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30257.log 2024-09-22T03:32:15.395 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29402.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29402.log.gz 2024-09-22T03:32:15.408 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31180.log 2024-09-22T03:32:15.408 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30257.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.30257.log.gz 2024-09-22T03:32:15.421 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29403.log 2024-09-22T03:32:15.421 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31180.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31180.log.gz 2024-09-22T03:32:15.433 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35800.log 2024-09-22T03:32:15.434 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29403.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29403.log.gz 2024-09-22T03:32:15.442 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34314.log 2024-09-22T03:32:15.453 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44365.log 2024-09-22T03:32:15.454 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34314.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34314.log.gz 2024-09-22T03:32:15.455 INFO:teuthology.orchestra.run.smithi146.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35800.log.gz 2024-09-22T03:32:15.462 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56944.log 2024-09-22T03:32:15.473 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49695.log 2024-09-22T03:32:15.473 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.56944.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56944.log.gz 2024-09-22T03:32:15.475 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44365.log.gz 2024-09-22T03:32:15.481 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36261.log 2024-09-22T03:32:15.483 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49695.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49695.log.gz 2024-09-22T03:32:15.490 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57952.log 2024-09-22T03:32:15.499 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44267.log 2024-09-22T03:32:15.503 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57952.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36261.log.gz 2024-09-22T03:32:15.507 INFO:teuthology.orchestra.run.smithi146.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57952.log.gz 2024-09-22T03:32:15.514 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43960.log 2024-09-22T03:32:15.515 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44267.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44267.log.gz 2024-09-22T03:32:15.534 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-09-22T03:32:15.534 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43960.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43960.log.gz 2024-09-22T03:32:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38177.log 2024-09-22T03:32:15.549 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42666.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gz 2024-09-22T03:32:15.557 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44147.log 2024-09-22T03:32:15.571 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38177.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38177.log.gz 2024-09-22T03:32:15.571 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30277.log 2024-09-22T03:32:15.571 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44147.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44147.log.gz 2024-09-22T03:32:15.579 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32704.log 2024-09-22T03:32:15.592 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30277.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30277.log.gz 2024-09-22T03:32:15.594 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43869.log 2024-09-22T03:32:15.595 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32704.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32704.log.gz 2024-09-22T03:32:15.603 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32620.log 2024-09-22T03:32:15.616 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43869.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43869.log.gz 2024-09-22T03:32:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58414.log 2024-09-22T03:32:15.618 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32620.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32620.log.gz 2024-09-22T03:32:15.628 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53852.log 2024-09-22T03:32:15.629 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.58414.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58414.log.gz 2024-09-22T03:32:15.637 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33824.log 2024-09-22T03:32:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53852.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53852.log.gz 2024-09-22T03:32:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57804.log 2024-09-22T03:32:15.651 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33824.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33824.log.gz 2024-09-22T03:32:15.659 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35391.log 2024-09-22T03:32:15.672 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.57804.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57804.log.gz 2024-09-22T03:32:15.675 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-09-22T03:32:15.676 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35391.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35391.log.gz 2024-09-22T03:32:15.684 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28679.log 2024-09-22T03:32:15.690 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.47117.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-09-22T03:32:15.693 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43820.log 2024-09-22T03:32:15.703 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.28679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41075.log 2024-09-22T03:32:15.706 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43820.log: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28679.log.gz 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr: 89.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49457.log 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.41075.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41075.log.gz 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49457.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49457.log.gz 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.43700.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38088.log 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.50844.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37129.log 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38088.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38088.log.gz 2024-09-22T03:32:15.725 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29764.log 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37129.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37129.log.gz 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43820.log.gz 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52002.log 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29764.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29764.log.gz 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54111.log 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.52002.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52002.log.gz 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55300.log 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54111.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54111.log.gz 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.55300.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55300.log.gz 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46822.log 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46416.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2024-09-22T03:32:15.726 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34762.log 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46822.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46822.log.gz 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54069.log 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34762.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34762.log.gz 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40160.log 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.54069.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54069.log.gz 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44946.log 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.40160.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40160.log.gz 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37038.log 2024-09-22T03:32:15.727 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.44946.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44946.log.gz 2024-09-22T03:32:15.738 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37870.log 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37038.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37038.log.gz 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36630.log 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.37870.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37870.log.gz 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45677.log 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36630.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36630.log.gz 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42132.log 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.45677.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45677.log.gz 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53236.log 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.42132.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42132.log.gz 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49228.log 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.53236.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53236.log.gz 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46576.log 2024-09-22T03:32:15.739 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.49228.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49228.log.gz 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38625.log 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.46576.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46576.log.gz 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32172.log 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.38625.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38625.log.gz 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29973.log 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32172.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32172.log.gz 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33292.log 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29973.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29973.log.gz 2024-09-22T03:32:15.740 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33292.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33292.log.gz 2024-09-22T03:32:15.820 INFO:teuthology.orchestra.run.smithi146.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.54403.log.gz 2024-09-22T03:32:15.990 INFO:teuthology.orchestra.run.smithi146.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.29996.log.gz 2024-09-22T03:32:16.109 INFO:teuthology.orchestra.run.smithi146.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.48150.log.gz 2024-09-22T03:32:16.176 INFO:teuthology.orchestra.run.smithi146.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.42560.log.gz 2024-09-22T03:32:16.388 INFO:teuthology.orchestra.run.smithi146.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.35946.log.gz 2024-09-22T03:32:48.489 INFO:teuthology.orchestra.run.smithi148.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-22T03:32:50.208 INFO:teuthology.orchestra.run.smithi146.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-22T03:32:50.900 INFO:teuthology.orchestra.run.smithi148.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-22T03:32:50.902 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-22T03:32:50.902 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m36.219s 2024-09-22T03:32:50.902 INFO:teuthology.orchestra.run.smithi148.stderr:user 1m7.457s 2024-09-22T03:32:50.902 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m2.502s 2024-09-22T03:32:51.505 INFO:teuthology.orchestra.run.smithi146.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-22T03:32:51.506 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-22T03:32:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m36.827s 2024-09-22T03:32:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:user 1m16.765s 2024-09-22T03:32:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m3.189s 2024-09-22T03:32:51.507 INFO:tasks.ceph:Archiving logs... 2024-09-22T03:32:51.507 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732/remote/smithi146/log 2024-09-22T03:32:51.508 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T03:33:03.578 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732/remote/smithi148/log 2024-09-22T03:33:03.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T03:33:14.223 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-22T03:33:14.231 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-22T03:33:14.232 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T03:33:14.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T03:33:14.323 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-22T03:33:14.673 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-22T03:33:14.674 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:33:14.677 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:33:14.691 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:33:14.694 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:33:14.696 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:33:14.696 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-22T03:33:14.696 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 -170us[ -170us] +/- 52ms 2024-09-22T03:33:14.696 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 +570us[ +570us] +/- 76ms 2024-09-22T03:33:14.696 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -305us[ -305us] +/- 60ms 2024-09-22T03:33:14.696 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:33:14.698 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:33:14.699 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-22T03:33:14.699 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 -160us[ -145us] +/- 52ms 2024-09-22T03:33:14.699 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 +579us[ +593us] +/- 76ms 2024-09-22T03:33:14.699 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 7 -283us[ -269us] +/- 60ms 2024-09-22T03:33:14.699 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:33:14.700 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-22T03:33:14.719 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-22T03:33:14.719 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-22T03:33:14.795 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-22T03:33:14.876 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-22T03:33:14.944 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:33:14.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:33:14.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:33:15.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:33:15.005 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-22T03:33:15.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-22T03:33:15.069 INFO:teuthology.task.internal:Duration was 1787.087427 seconds 2024-09-22T03:33:15.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-22T03:33:15.108 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-22T03:33:15.108 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T03:33:15.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T03:33:15.150 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:33:15.151 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:33:15.445 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-22T03:33:15.446 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-09-22T03:33:15.446 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T03:33:15.507 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-09-22T03:33:15.507 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T03:33:15.534 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-22T03:33:15.534 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T03:33:15.549 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T03:33:15.600 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-22T03:33:15.601 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T03:33:15.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T03:33:15.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-22T03:33:15.759 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-22T03:33:15.760 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T03:33:15.784 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T03:33:15.810 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-22T03:33:15.837 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:33:15.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:33:15.860 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-09-22T03:33:15.874 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-09-22T03:33:15.900 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:33:15.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:33:15.954 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:33:15.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:33:15.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-22T03:33:15.978 INFO:teuthology.task.internal:Transferring archived files... 2024-09-22T03:33:15.978 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732/remote/smithi146 2024-09-22T03:33:15.979 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T03:33:16.028 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914732/remote/smithi148 2024-09-22T03:33:16.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T03:33:16.059 INFO:teuthology.task.internal:Removing archive directory... 2024-09-22T03:33:16.059 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T03:33:16.064 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T03:33:16.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-22T03:33:16.121 INFO:teuthology.task.internal:Not uploading archives. 2024-09-22T03:33:16.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-22T03:33:16.146 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-22T03:33:16.146 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T03:33:16.148 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T03:33:16.160 INFO:teuthology.orchestra.run.smithi146.stdout: 265613 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 03:33 /home/ubuntu/cephtest 2024-09-22T03:33:16.165 INFO:teuthology.orchestra.run.smithi148.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 03:33 /home/ubuntu/cephtest 2024-09-22T03:33:16.166 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-22T03:33:16.179 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-22T03:33:16.248 INFO:teuthology.run:Summary data: description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} duration: 1787.087426662445 flavor: crimson owner: scheduled_teuthology@teuthology success: true 2024-09-22T03:33:16.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T03:33:16.343 INFO:teuthology.run:pass